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ABSTRACT 

Nearly  6,000  TRICARE  beneficiaries  per  annum  receive  outpatient  medical  care  in  one  of  24 
Korean  hospitals  within  the  Republic  of  Korea  (ROK).  The  objective  of  this  project  was  to 
determine  TRICARE  U.S.  outpatient  beneficiary  satisfaction  at  Memorandum  of  Understanding 
(MOU)  Host  Nation  Hospitals  (HNH)  in  the  Republic  of  Korea.  The  point  of  service  survey 
used  for  this  project  was  aligned  with  the  Department  of  Defense  (DoD)  MHS’s  Army  Provider 
Level  Satisfaction  Survey  (APLSS).  The  survey  utilizes  a  Likert  5-point  scale  system.  The 
setting  included  medical  facilities  within  the  ROK  that  participate  under  a  MOU  with  the  18lh 
Medical  Command.  The  sample  included  TRICARE  beneficiaries  referred  for  outpatient  care  to 
Samsung,  Dongsan,  and  Dankook  hospitals,  from  September-November  2007.  The  analysis 
measured  overall  patient  satisfaction  as  it  relates  to  beneficiary  category,  gender  and  command 
sponsorship.  The  project  data  was  utilized  to  verify  and/or  identify  potential  target  areas  of 
patient  satisfaction. 
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INTRODUCTION 

Background 

The  Command  Structure  and  Population 

There  are  more  than  80  Army  installations,  18  Air  Force  sites,  and  two  Navy  facilities  with 
an  estimated  37,500  US  military  personnel  assigned  throughout  the  98,480  sq  km  Republic  of 
Korea  (ROK)  (U.S.  Forces  Korea.  2007;  South  Korea,  2007).  There  are  three  major  commands 
with  the  ROK.  The  United  Nations  Command  (UNC)  has  the  mission  to  maintain  the  provisions 
of  the  truce  or  Armistice  Agreement,  established  27  July  1953  between  North  and  South  Korea. 
The  Combined  Forces  Command  (CFC)  has  the  mission  to  "Deter  hostile  acts  of  external 
aggression  against  the  Republic  of  Korea  by  a  combined  military  effort  of  the  United  States  of 
America  and  the  ROK;  and  in  the  event  deterrence  fails,  defeat  an  external  armed  attack  against 
the  ROK”  (U.S.  Forces  Korea,  2007.  p.  1).  Finally,  the  United  States  Forces  Korea  Command 
(USFK),  which  commands  all  United  States  Forces  in  Korea,  which  includes  five  component 
commands  with  over  1 7  subordinate  commands.  The  USFK  Commander  serves  as  the 
Commander  of  the  United  Nations  Command  and  Combined  Forces  Command  (Global  Security, 
2007). 

The  largest  component  command  within  the  ROK  is  the  Eighth  United  States  Army  (EUSA). 
Its  mission  is  to  “support  deterrence  of  North  Korea  aggression  against  the  ROK.  Should 
deterrence  fail.  Eighth  United  States  Army  supports  Non-combatant  Evacuation  Operations 
(NEO),  transitions  to  hostilities,  generates  combat  power  to  support  UNC  and  CFC 's  campaign, 
and  provides  combat  support  and  combat  service  support  to  assigned,  attached,  and  other 
designated  forces  within  the  Korean  Theater  of  Operations.  On  order,  conducts  combat 
operations”  (Mission  of  the  EUSA,  2006,  p.  1). 
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The  Korean  peninsula  is  divided  into  several  geographic  areas,  as  seen  in  Appendix  A,  with 
the  majority  of  personnel  residing  in  the  Seoul-Yongsan  area  (approximately  1 1,400  according  to 
Personnel  Information  Management  System  Korea  (PIMSK,  2007).  This  area  included  the 
largest  portion  of  command  sponsored  family  members  (nearly  70%  or  2,413  of  the  3,504 
command  sponsored  family  members-  (PIMSK,  2007).  The  command  sponsored  program  helps 
to  stabilize  critical  positions  within  the  ROK  by  authorizing  family  members  to  accompany  the 
service  member  and  receive  full  benefits  (e.g.,  housing,  school,  medical,  dental)  throughout  their 
ROK  tour  of  duty.  As  of  April  2007,  there  are  approximately  53,000  personnel  within  the  ROK 
in  which  the  USFK  has  the  responsibility  to  ensure  medical  care  is  both  available  and  accessible 
(PIMSK,  2007).  Of  the  53,000  personnel,  32,000  active  duty  and  command  sponsored  active 
duty  family  members  are  enrolled  in  the  military  health  system's  health  plan,  TRICARE.  There 
are  an  estimated  23.000  personnel  (including  non-command  sponsored  active  duty  family 
members.  Department  of  Defense  (DoD)  civilians,  contractors,  retirees  and  their  families)  within 
the  ROK,  not  enrolled  in  TRICARE. 

Healthcare  Delivery 

The  18th  Medical  Command,  one  of  the  17  subordinate  commands,  is  also  accountable  to  be 
aligned  with  the  military  health  system’s  mission:  “To  enhance  DoD  and  our  Nation's  security 
by  providing  health  support  for  the  full  range  of  military  operations  and  sustaining  the  health  of 
all  those  entrusted  to  our  care”  (Office  of  the  Assistant  Secretary  of  Defense  for  Health  Affairs, 
2003,  p.  1).  As  such,  the  mission  of  the  18lii  Medical  Command  is  to  “Ensure  integrated  and 
comprehensive  Theater  Health  Support  (THS)  to  all  beneficiaries  throughout  the  Korean  Theater 
and  across  the  entire  spectrum  of  operations”  (18th  MEDCOM  mission,  2003,  p.  1).  The  18th 
Medical  Command’s  subordinate  units  throughout  the  ROK  include:  the  121st  Combat  Support 
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Hospital  (CSH),  the  168th  Area  Support  Medical  Battalion  (ASMB),  the  618th  Dental  Company, 
the  16th  Medical  Logistics  Battalion,  a  veterinarian  detachment  and  several  medical  detachments. 
Together,  121st  CSH  and  168th  ASMB  are  identified  as  the  Integrated  Healthcare  Organization 
(IHO)  responsible  for  the  healthcare  delivery  throughout  the  entire  ROK. 

The  121'1  CSH  is  a  community-type  hospital  with  the  dual  missions  of  supporting  field 
medicine  and  community-based  medicine  throughout  the  ROK.  The  1214  CSH  has  an  in-patient 
capacity  of  over  60  beds  with  outpatient  services  to  include:  medical,  surgical,  obstetric, 
gynecological,  pediatric  and  psychiatric  care.  Care  is  provided  to  all  active  duty  personnel  and 
their  family  members,  as  well  as,  government-employed  civilians,  contractors  and  their  family 
members.  Many  specialty  services  are  not  available;  those  requiring  medical  care  beyond  the 
capabilities  of  the  12 14  CSI 1  are  referred  to  Host  Nation  Hospitals  (HNH)  and/or  evacuated  to 
other  military  hospitals  outside  the  ROK.  The  IHO  is  comprised  of  approximately  2,600  staff 
members  (1,900  military,  450  government  service  (GS),  Korean  government  service  and 
volunteers,  210  Korean  Service  Corps  and  40  contract  staff  (Jolissaint,  2007).  In  fiscal  year 
2007,  the  IHO  had  302,346  total  outpatient  visits,  averaging  over  1,100  visits  per  day,  further 
broken  down  into  beneficiary  categories  in  Figure  1. 
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IHO  Beneficiary  Visits  FY07 


Dependent 

Beneficiary  Category 


Figure  1.  IHO  FY2007  total  outpatient  patient  visits  (M2,  2  March  2008). 

Top  outpatient  services  by  volume  for  the  IHO  are:  Primary  Care.  Physical  Therapy, 
Emergency  Care,  Occupational  I  Iealth  and  Optometry.  Inpatient  top  services  include: 
Orthopedics,  General  Surgery,  OB/GYN  and  ENT  (M2  data-pull,  April  2007).  The  units  that 
comprise  the  IHO  participate  in  numerous  community  activities  including  health  fairs,  town  hall 
meetings,  well-being  councils,  school  support,  and  alliance  health  programs  with  Republic  of 
Korea.  The  IHO  is  a  key  participant  for  several  resource  demanding  (i.e.,  medical  staffing 
requirements)  major  exercises  and  training  events  each  year.  The  IIIO's  mission  is  both 
complex  and  demanding.  A  typical  workload  for  the  IHO  during  a  duty  day  can  be  seen  in 
Figure  2. 
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Figure  2.  Typical  daily  workload  within  the  IHO  (Jolissaint,  2007). 

The  Health  Plan  (TRICARE) 

The  military  health  system’s  unique  health  plan  for  military  members  and  their  dependents  is 
called  TRICARE.  TRICARE  manages  this  healthcare  plan  in  coordination  with  organic  military 
medical  infrastructure  along  with  a  vast  worldwide  network  of  civilian  counterparts.  As  of 
January  2007,  TRICARE  has  over  nine  million  eligible  beneficiaries,  65  military  hospitals,  412 
medical  clinics  and  over  400  dental  clinics  worldwide  (TMA,  2007).  TRICARE  benefits  have 
continued  to  increase  since  its  inception.  As  of  April  2006,  the  TRICARE  network  includes 
nearly  250,000  physicians,  every  U.S.  hospital  and  55,000  retail  pharmacies.  An  independent 
survey  shows  TRICARE  ranks  as  one  of  the  Nation’s  best  health  plans  (Quadrennial  Defense 
Review,  2006). 

Many  military  treatment  facilities,  especially  overseas,  are  continually  becoming  unable  to 
meet  the  demand  of  the  beneficiaries  (primary  and  specialty  care)  due  to  limited  resources  and 
mission  priorities  (e.g.,  the  military  has  a  unique  wartime  mission).  TRICARE  and  its  managed 
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care  business  practices  (responsible  for  the  care  of  its  beneficiaries,  regardless  of  location)  allows 
for  patients  to  be  referred  outside  the  military  treatment  facility  for  care.  In  the  overseas 
environment  this  implies  that  patients  are  and  will  continue  to  be  referred  to  HNH.  Current 
agreements  with  TRICARE  and  HNH  require  families  that  are  non-command  sponsored  within 
the  ROK  to  pay  all  upfront  medical  expenses  when  care  is  received  through  a  host  nation 
hospital  (Philpott,  2006).  In  many  cases  this  causes  financial  hardship  for  families  and 
satisfaction  challenges  for  the  healthcare  plan.  “The  amount  of  money  a  patient  has  to  incur  out 
of  pocket  has  a  direct  influence  on  their  assessment  of  healthcare  received’’  (Aday,  1996,  p.  15). 
The  Korean  Healthcare  System 

The  ROK  is  a  heavily  populated  country  resulting  in  high  levels  of  congestion  in  the  larger 
cities.  According  to  the  Population  Reference  Bureau,  the  ROK  has  an  extremely  high 
population  density  of  487  people  per  square  kilometer  compared  to  3 1  people  per  square 
kilometer  in  the  United  States  (Population  Reference  Bureau,  2007).  A  few  additional  facts  on 
the  ROK’s  healthcare,  as  seen  in  Appendix  B  include:  66  beds  per  10,000  people  (33  U.S.),  1.57 
physicians  (2.56  U.S.)  and  1.75  nurses  (9.37  U.S.)  per  1000  people  (Population  Reference 
Bureau,  2007;  World  Health  Organization,  2006).  Throughout  the  ROK,  medical  care  is 
delivered  and  managed  \  ia  a  National  Health  Insurance  system.  “The  quality  of  Korean 
healthcare  is  excellent  and  many  healthcare  providers  speak  English”  (United  States  Naval 
hospital,  2007,  p.  1 ).  1  lealthcare  delivery  within  the  ROK  is  accomplished  via  a  three-tiered 
system  based  upon  bed  capacity  and  capabilities  of  the  facility  (i.e.,  third  tier  includes  facilities 
with  greater  than  700  beds)  (Cho,  Lee,  Kim,  Lee  &  Choi,  2004).  Koreans  have  free  access  to 
any  first  or  second  tier  medical  facility,  to  include  providers  and  specialists  working  in  those 
facilities;  referrals  are  required  for  tier  three  facilities  (Choi,  Kim  &  Lee,  1998).  As  such,  the 
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Korean  healthcare  delivery  system  lacks  a  primary  care  base,  which  leads  to  many  efficiency  and 
patient  satisfaction  challenges.  In  an  attempt  to  address  these  challenges,  several  quality 
improvement  initiatives  within  the  ROK  were  developed  (e.g.,  establishing  Quality'  Management 
Departments).  In  addition  to  the  ROK  being  a  male-dominated  society,  there  are  many  cultural 
differences  when  compared  to  the  U.S.  healthcare  delivery  system.  Within  the  ROK,  families  play 
a  critical  role  in  the  care  and  recovery  of  their  hospitalized  family  members.  Family  members  stay 
an  extended  period  of  time  taking  care  of  their  loved  ones,  doing  many  of  the  daily  living 
activities/support  a  licensed  practical  nurse  (LPN)  may  do  in  the  U.S.  There  are  no  LPNs  and  by 
design  a  short  supply  of  nurses  for  this  reason.  Many  of  the  large  medical  facilities  in  the  ROK 
were  designed  via  U.S.  architectural  firms  with  the  intent  to  make  the  hospital,  less  like  a  hospital. 
Facilities  are  designed  to  have  a  community  feel  with  social,  food  and  shopping  mall  areas.  The 
majority  of  Korean  hospitals  are  on  the  cutting  edge  of  leveraging  information  technology  in  the 
healthcare  environment.  Korean  hospitals  utilize  a  paperless,  electronic  medical  record  system, 
however  U.S.  patient  information  is  not  accessible  in  either  direction;  hard  copy  information  is  the 
primary  means  of  documenting,  recording  and  transmitting  (back  to  the  IHO)  a  U.S.  patient’s 
medical  information  in  the  HNH. 

HNH  Memorandums  of  Agreement 

The  demanding  workload  and  pace  within  the  Korean  theater  of  operations,  coupled  with  the 
large  beneficiary  population  results  in  a  continuous  challenge  to  retain  sick  or  injured 
beneficiaries  within  the  theater.  “There  were  daily  medical  evacuation  flights  costing  vast 
amounts  of  money  and  lost  productivity  (and  operational  readiness),  prior  to  having  the  MOUs” 
(Choi,  2007,  p.  1).  Challenges  to  decrease  the  numerous  and  costly  medical  evacuations 
continue  to  be  a  priority  in  healthcare  delivery  within  the  theater.  TRICARE  has  as  an 
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established  and  published  set  of  aeeess  to  eare  standards.  Congress  mandated,  under  the  Defense 
Authorization  Act  of  1 999.  that  the  military  health  system  meet  the  following  access  to  care 
standards  for  its  beneficiaries:  aeute  care  within  24  hours,  routine  eare  within  one  week,  specialty 
eare  within  four  w'eeks  and  wellness  care  within  four  weeks  (Corey,  1997). 

In  order  to  meet  the  demands  of  the  beneficiary  population,  establish  billing  procedures  and 
remain  within  TRICARE  aeeess  to  care  standards,  the  Memorandum  of  Understanding  (MOU) 
program  was  established  with  ROK  1  INI  Is.  The  ROK  MOU  program  allows  for  those  U.S. 
benefieiaries  under  TRICARE  Prime  to  seek  care  without  the  concern  of  upfront  expenses; 
TRICARE  will  pay  the  bill  once  it  is  processed  and  received.  Requirements  for  TRICARE 
Standard  (non-eonnnand  sponsored)  benefieiaries  require  full  payment  at  the  time  of  discharge. 
The  MOU  program  began  on  Oetober  28.  1999  with  DanKook  University  Medical  Center  in 
Pyungtaek,  South  Korea  and  continues  today,  with  the  24th  MOU  established  with  Yeungnam 
University  Medieal  Center  in  Daegu.  South  Korea  on  17  August  2007.  Many  of  the  MOU  HNHs 
are  teaching  hospitals  with  the  latest  medical  techniques  and  technologies.  These  MOU  facilities 
are  located  throughout  the  Korean  Peninsula  to  ensure  a  continuity  of  care  as  seen  in  Appendix 
A.  These  MOU  facilities  take  IHO  overflow  eases  and  emergent  care  when  needed,  but  the 
referral  demand  is  primarily  specialty  care  (i.e..  Neurosurgery,  Cardiology  and  Urology)  not 
available  via  the  IHO.  Top  outpatient  referrals  to  MOU  facilities  inelude:  MRI,  Urology, 
Ultrasound,  Obstetries  and  Physieal  Therapy  (Patriek,  2007). 

The  MOU  program  continues  to  save  money,  inerease  benefieiary  morale  (i.e.,  town  hall 
meetings  and  patient  feedback),  increase  operational  readiness  (retaining  personnel  loeally),  and 
return  personnel  to  duty  in  a  short  period  of  time  (Choi,  2007).  The  overseas  military  health 
system,  due  to  demand  and  resources,  has  and  will  continue  to  laek  many  in-house  specialty 
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services,  for  this  reason,  such  services  will  continue  to  be  referred  out  to  HNH.  Forgoing  costly 
and  lengthy  travel  for  care  has  saved  the  military  health  system  millions  of  dollars  (Allen  & 
Little,  2007).  The  MOU  agreements  clearly  establish  responsibilities  and  coordination 
requirements  for  both  the  IHO  and  the  1 INH.  These  agreements  state  several  requirements 
including:  quality  assurance,  patient  assessments  and  various  surveys.  HNH  are  carefully  chosen 
based  on  their  capabilities  and  the  needs  of  the  beneficiary  populations  they  may  serve.  Officials 
in  both  regions  agree  that  the  HNH  have  repeatedly  done  an  excellent  job  (Allen  &  Little,  2007; 
Choi,  2007).  Host  Nation  Hospitals  are  inspected  and  approved  via  the  IHO  Deputy  Commander 
for  Clinical  Services  (C'MO)  and  the  Commander  (CEO)  of  the  IHO.  Compliance  inspections 
take  place  throughout  the  year  on  a  scheduled  basis. 

Conditions  Which  Prompted  the  Project 

The  Department  of  Defense’s  Military  Health  System  has  set  patient  satisfaction  targets  for 
all  military  treatment  facilities,  however  when  U.S.  outpatients  are  cared  for  outside  a  military 
treatment  facility  (as  in  the  Republic  of  Korea)  patient  satisfaction  has  not  been  successfully 
monitored  or  measured.  In  November  of  2000,  the  IHO  developed  and  implemented  a  survey  to 
capture  and  report  patient  satisfaction  data  for  those  U.S.  beneficiaries  receiving  care  as  an 
outpatient  in  one  of  the  several  HNHs.  The  MOU  HNH  satisfaction  survey,  as  seen  in  Appendix 
C,  historically,  has  experienced  a  poor  response  rate  (<  4%)  and  did  not  appropriately  collect, 
analyze  and  monitor  the  data  for  appropriate  action  by  healthcare  leadership  (Patrick,  2007). 
“When  a  Military  Health  System  (MHS)  beneficiary.  USFK  employee,  or  family  member  is  sent 
to  a  Korean  hospital,  our  most  significant  concern  is  the  quality  of  care  they  will  receive"  (18th 
Medical  Command  MOU  Hospitals.  2007,  p.  1).  Beneficiaries  are  promised  a  predetermined 
level  of  cost,  access,  and  quality  healthcare  by  their  health  plan.  TRICARE.  In  order  to  ensure 
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the  health  system  is  compliant,  data  must  continuously  he  collected  and  analyzed.  This  data  is 
additionally  required  for  the  development  of  future  managed  care  contracts.  Patient  satisfaction 
requires  having  a  system  to  continuously  identify  beneficiaries'  perceptions  and  expectations 
(further  defining  this  construct,  patient  satisfaction)  allowing  leadership  and  planners  to  react 
appropriately.  Mangclsdorff  suggested  patient  satisfaction  as  a  criterion  for  measuring  quality  of 
care  (Mangel sdorff.  1 979).  Patients  are  becoming  more  and  more  informed  on  medical 
intervention  and  deli\  cry.  Patients  understand  what  board  certification  and  accreditation  implies 
of  a  medical  treatment  provider  or  facility;  this  information  is  linked  to  quality  care  (Roizen,  Oz, 
&  Joint  Commission.  2006).  Patient  satisfaction  is  considered  an  outcome  of  quality  care  (Cho, 
Lee,  Kim.  Lee  &  Choi.  2004). 

The  military’s  health  system  is  under  ever  increasing  pressure  to  provide  quality  patient  care 
within  established  access  standards  while  simultaneously  reducing  its  expenditures.  Political 
scrutiny  continues  to  be  placed  on  the  expenditures  and  outputs  of  the  system.  The  private  sector 
medical  facilities  are  ready  to  take  over  the  majority  of  the  medical  delivery  mission.  Overseas 
medical  treatment  facilities  share  the  same  concerns;  however  they  encounter  unique  challenges 
due  to  their  location  and  operating  environment  (e.g.,  lack  of  a  managed  care  contract,  language 
challenges,  limited  resources  and  access  of  specific  services). 

Private  businesses  must  understand  the  needs  and  expectations  of  its  customers  and  the 
military  health  system  is  no  different.  Customers,  especially  in  healthcare,  are  the  driving  force 
in  formulating  the  needs  of  an  organization  to  include,  staffing  and  financing.  It  is  understood 
that  the  1HO  alone  does  not  have  the  infrastructure  or  services  to  ensure  access  to  care  standards 
are  met  for  all  eligible  beneficiaries  (current  and  future).  Monitoring  and  more  importantly 
understanding  militan  patients'  satisfaction  with  the  delivery  of  healthcare  has  become  a  crucial 
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part  in  comparing  both  military  and  non-military  (which  continue  to  grow)  providers  that  are  a 
part  of  the  TRICARE  program  (Tucker  &  Adams,  2001).  TRICARE  requires  the  IHO  to 
maintain  the  same  standards  (including  access  to  care  standards)  as  military  medical  facilities 
within  the  states,  which  serves  as  a  dri\  ing  force  to  obtain  and  maintain  HNH  MOUs.  As  the 
stabilization  movement  continues  in  the  Republic  of  Korea  (e.g.,  increasing  command  sponsored 
authorizations),  the  healthcare  needs,  primarily  specialty  care,  of  its  beneficiaries  will  continue  to 
grow.  Monitoring  beneficiaries’  perceptions,  needs  and  expectations  may  provide  valuable 
information  to  IHO  and  1 1NH  leadership.  This  information  may  provide  additional  insight  about 
the  host  nation  medical  provider;  do  they  understand  the  U.S.  population’s  culture,  expectations 
and  perceptions  about  health  care  and  its  delivery. 

Military  medical  treatment  facilities  (MTF)  must,  now  more  the  ever,  closely  monitor  patient 
satisfaction  in  order  to  meet,  maintain  and  change  accordingly  with  their  beneficiaries’  needs. 
This  is  a  critical  portion  of  all  MTF  strategic  and  financial  planning.  The  Department  of 
Defense’s  Military  I  Iealth  System  has  ongoing  patient  satisfaction  surveys.  One  is  an  Inpatient 
Customer  Satisfaction  Survey  for  inpatients  receiving  care  at  a  military  treatment  facility  and  the 
other,  a  Purchased  Care  Outpatient  Visit  Satisfaction  Survey  (or  APLSS  as  seen  in  Appendix  D), 
for  outpatients  receiving  care  at  a  network  or  contract  provider  outside  the  military  treatment 
facility.  As  this  project  focuses  on  the  outpatient,  APLSS  appears  as  method  of  choice  to  capture 
satisfaction  information.  However,  the  APLSS  it  is  not  utilized  within  the  ROK  to  collect 
satisfaction  data  from  U.S.  outpatients  referred  to  MOU  HNHs  (New  Surveys,  2000).  The  MOU 
HNH  satisfaction  survey's  success  is  critical  for  managing  beneficiary  satisfaction  when 
beneficiaries  receive  medical  care  outside  the  military  MTF.  T  he  IHO  needs  this  data  to 
adequately  measure  and  monitor  their  entire  population.  This  patient  satisfaction  data  may 
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provide  an  excellent  quality  indicator(s)  for  future  planning  of  the  capabilities  and  services 
offered  by  the  IHO  with  ability  to  compare  with  similar  APLSS  data. 

The  ICE  or  Interactive  Customer  Evaluation  used  throughout  the  Department  of  Defense 
(DoD)  allows  customers  to  rate  products  and  services  provided  by  DoD  offices  and  facilities 
worldwide.  The  ICE  survey  questions,  prior  to  December  2007,  were  similar  to  the  IHO  and 
APLSS  surveys,  including  the  5-point  Likert  scales.  Although  the  ICE  and  APLSS  collect 
patient  satisfaction  data,  they  do  not  communicate  with  one  another.  The  IHO  Patient  Advocate 
monitors  the  ICE  system  as  well  as  the  APLSS  system.  The  Patient  Advocate  collects  data  from 
both  systems  independently,  along  with  the  internal  comment  cards  in  order  to  report  to  the  IHO 
leadership  monthly  and  on  an  as  needed  basis.  Prior  to  December  2007,  the  ICE  Host  Nation 
Referral  Comment  Card  (Appendix  E)  has  experienced  a  poor  response  rate,  with  less  than  200 
responses  over  the  past  five  years  (out  of  an  estimated  30,000  referrals)  (Robbins,  2007).  In 
December  2007  the  1(T  1  lost  Nation  Referral  Comment  Card  was  updated  with  the  same 
questions  (as  the  MOU  1  INI  1  Survey)  to  supplement  the  hard  copy  survey,  while  providing  an 
additional  means  (online)  for  beneficiaries  to  report  patient  satisfaction. 

Ensuring  an  organization's  customer  base  is  and  remains  satisfied  is  crucial  in  order  to  endure 
and  develop  in  today's  complex  environment;  the  business  of  healthcare  is  no  different.  The 
Commander  of  the  1110  continues  to  place  patient  satisfaction  as  a  priority  within  the  IHO 
organization,  stating  in  all  staff-newcomer  orientations  that  one  of  the  IHO's  priorities  is  to  bring 
about  a  positive  reaction  in  all  our  customers  (Jolissaint,  2007).  Dissatisfied  patients  often  hold 
critical  improvement  information.  Healthcare,  more  than  any  industry,  must  continuously 
determine  how  satisfied  its  customers  arc,  view  trends,  establish  changes,  and  assess  changes 
made.  Leaders  can  truly  capitalize  knowing  what  is  important  to  customers  by  their  feedback. 
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Statement  of  the  Problem 

The  IHO  HNH  survey  was  not  being  effectively  utilized  in  the  collection,  analysis,  reporting 
and  monitoring  of  referred,  outpatient  U.S.  beneficiary's  satisfaction  after  receiving  care  at  a 
HNH.  Although  the  survey  exists,  the  surveys  themselves  have  fallen  short  in  advertising  to  the 
beneficiaries  as  an  available  means  to  report  their  satisfaction.  Surveys,  prior  to  this  project,  that 
were  collected  were  unsuccessful  in  ha\  ing  their  data  collected,  reviewed,  analyzed  and  reported. 
The  MOU  HNH  survey  must  be  re-energized  and  marketed  to  the  beneficiary  population.  This 
data  is  critical  in  order  to  determine  whether  or  not  1 1NH  beneficiary  satisfaction  meets  the 
DoD’s  and  military  health  system's  95%  (satisfied)  target  for  overall  satisfaction  with  provider. 
Additionally,  the  MOU  I  INI  1  survey  may  provide  needed  data  to  determine  and  track  U.S. 
beneficiary  overall  satisfaction  with  HNH  facility  visit.  Timely  and  statistically  analyzed  (and  in 
accordance  with  the  organization's  needs)  satisfaction  data  may  improve  healthcare  delivery  to 
our  beneficiaries  and  working  relationships  between  the  IHO  and  the  HNH;  this  is  also  a 
requirement  under  the  current  MOU  agreements.  Delivering  the  full  scope  of  healthcare  to  our 
beneficiary  population  in  the  ROK  continually  proves  to  be  a  challenge  given  the  allocated 
resources  provided  to  the  military  healthcare  system  within  the  Korean  Peninsula.  Personnel 
shortages,  inability  to  provide  the  full  range  of  specialty  care,  the  geographical  location,  and  the 
continued  scrutiny  of  healthcare  expenditures  create  obstacles  for  the  quality  and  access  the 
military  health  system  promises  its  beneficiaries.  As  the  demand  for  outpatient  services 
continues,  the  requirement  to  maintain  MOUs  with  I INH  becomes  a  critical  piece  in  the  delivery 
of  healthcare  in  Korea.  As  seen  in  Appendix  F,  the  challenge  to  maintain  TRICARE  access 
standards  for  the  ROK  beneficiary  population  is  real  and  has  lead  to  several  measures,  all 
involving  a  close  and  continued  partnership  with  its  MOU  HNH.  Utilizing  HNH  does  not 
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release  the  military  health  system  of  this  TRICARE  promise.  The  IHO  has  an  internally 
designed  survey,  yet  it  is  missing  a  systematic  and  standardized  means  to  analyze  measure,  share 
and  appropriately  utilize  the  patient  satisfaction  data.  Historically,  the  survey  never  received 
greater  than  a  four  percent  response  rate  (Patrick,  2007).  Determining  differences  in  satisfaction 
amongst  beneficiary  categories  is  an  area  of  particular  focus.  Data  from  the  internal  survey  was 
rarely  compared  against  itself  to  demonstrate  potential  trends  or  indicators.  Additionally,  this 
data  was  not  shared  with  HNI1  leadership.  All  involved  parties  (IHO  and  HNH  leadership) 
require  this  data  to  identify  both  successful  and  unsuccessful  processes  and  systems  and 
incorporate  process  improvement  where  needed  (Patrick,  2007).  One  cannot  improve,  nor 
properly  monitor,  organizational  conditions,  objectives  and  goals  without  being  able  to  quantify 
and  measure  them.  Leadership  and  providers  do  not  have  timely  and  therefore,  typically  not 
actionable  feedback.  Without  feedback,  it  is  difficult  to  improve  the  system  and  care  received. 

Literature  Review- 

Patient  satisfaction  research,  studies,  recommendations,  definition,  designs  and  opinions  are 
clearly  abundant  throughout  the  world.  As  such,  it  is  understood  and  expected  that  differences 
exist  throughout  these  works.  Although  differences  do  exist,  this  literature  review  uncovered  a 
common  thread.  That  thread  of  similarity  is  found  in  the  intent  of  measuring,  monitoring  and 
reporting  this  data  and  that  collecting,  understanding  and  reacting  to  patients’  (customers’) 
needs,  wants,  perceptions  and  expectations  are  critical  to  future  growth  and  success  of  any 
organization.  Just  as  a  customer  decides  on  which  computer  system  to  purchase,  the  patient,  now 
more  than  ever,  has  control  to  make  healthcare  decisions;  providers’  only  influence  the  decision. 
Understanding  what  the  patient  expects  is  critical.  Satisfaction  is  tracked  in  nearly  every 
industry,  especially  within  the  United  States.  Korea  in  1995,  after  consumers,  healthcare 
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professionals  and  portions  of  the  Government  continued  pressures  on  hospitals  for  lack  of  quality 
of  care;  the  Government  instituted  the  Hospital  Services  Evaluation  Program.  The  outcome  of 
this  program  resulted  in  a  wave  of  quality  initiatives  to  include  the  establishment  of  Quality 
Departments  within  the  hospitals  (Cho  &  Kim.  2000).  In  a  1096  Korean  study,  nearly  all  of  the 
235  general  hospitals  examined  already  had  in  place  or,  for  those  who  did  not,  planned  to 
conduct  patient  satisfaction  surveys.  Those  that  planned  to  do  so  had  to  overcome  obstacles  such 
as:  lack  of  leader  interest,  lack  of  an  adequate  survey,  unable  to  confirm  reliability  and  validity 
and  lack  of  skill  sets  necessary  to  statistically  analyze  and  report  the  data  (Lee,  Kim,  Cho  &  Lee, 
1998).  The  Joint  Commission,  the  National  Committee  for  Quality  Assurance,  Health  Plan 
Employer  Data  and  Information  Set  (HED1S)-  which  has  several  measures  addressing 
satisfaction,  the  Centers  for  Medicare  and  Medicaid  (CMS),  the  International  Organization  for 
Standardization,  along  with  several  other  agencies  consider  patient  satisfaction  particularly 
important  (White,  1 999).  Healthcare  perceptions  and  expectations  of  patients  are  real;  they  do 
exist  and  may  influence  a  patient’s  overall  satisfaction  of  their  medical  care  and  experience. 

What  is  Patient  Satisfaction  and  What  Influences  It? 

Defining  satisfaction  continues  to  affect  our  quality-focused  health  care  culture.  An  expert  in 
the  field  of  quality  improvement  onee  stated  that  to  begin  measuring  an  item  without  firmly 
knowing  what  it  is  you  are  measuring  “is  to  court  disaster”  (Donabedian,  1988,  p.  1). 

Satisfaction  is  an  attitude  and  an  assessment  of  sen  ice  received.  “Its  main  purpose  is  to  identify 
problems  in  the  provision  of  earc  that  may  be  improved  by  managerial  intervention”  (Fitzpatrick 
&  Hopkins,  1993,  p.  3).  Providers’  value  patient  feedback,  providing  the  feedback  is  timely  and 
actionable  (Army  Office  of  the  Surgeon  General,  2007).  Patients  too,  value  feedback  and  timely 
information.  Robbins  (Robbins  ct  al„  1993)  discussed  a  strong  (positive)  correlation  with  the 
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amount  of  information  patients  received  from  their  providers  during  their  visit  and  their 
satisfaction.  A  purchaser  of  a  consumer  good  or  product  expects  a  high  level  of  customer  service 
throughout  the  buying  experience.  Patients  are  no  different;  they  too,  expect  a  high  level  of 
customer  service  throughout  the  continuity  of  care  received  (and  purchased).  This  consumer 
driven  market  is  pressuring  healthcare  leaders  to  exploit  all  the  potential  opportunities  it  has  to 
interact  with  its  patients  (customers)  to  ensure  a  total  positive/satisfied  experience  (Spiegelman 
&  Sensor,  2008). 

A  country’s  culture  may  influence  satisfaction;  in  Korea,  gender  and  age  are  key  determinants 
on  how  the  soeiety  cares  for  a  patient.  Healthcare  ean  be  viewed  from  several  perspectives.  The 
culture  within  Korea  (and  much  of  Asia)  is  strong  in  tradition  and  history;  it  holds  elders  in  high 
respeet  and  males  in  a  higher  status  than  females.  To  the  U.S.  benefieiary,  the  Asian  eulture  has 
an  influence  on  host  nation  medical  staff  and  standards  of  care.  For  example,  nursing  standards 
vary,  often  requiring  family  members  to  take  on  a  more  active  (compared  to  a  U.S.  hospital)  role 
in  the  care  and  recovery  of  their  family  member  (e.g..  feeding  and  bathing).  A  Naval  executive 
officer,  Captain  Gerald  R.  Cox  stated,  “Another  difference  is  the  attitudes  of  some  doctors, 
particularly  older  doctors,  who  may  seem  to  be  more  paternalistic  in  their  relationship  with 
patients”  (Allen  &  Little,  2007,  p.  3).  A  patient's  pain  management  and  food,  in  a  HNH,  are  key 
concerns  for  referring  military  doetors.  “For  pain  management... there  is  a  cultural  difference  in 
the  area  of  pain  perception  and  management,  but  we  are  steadily  making  improvement.  Air 
Force,  Lieutenant  Colonel  Catherine  Bard  stated,  “It  is  not  uncommon  for  patients  to  complain 
about  hospital  food  in  the  States... food  served  in  most  HNH  is  different,  but  it  is  nutritious” 
(Allen  &  Little,  2007,  p.  4). 
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The  provider-patient  relationship  is  the  most  influential  factor  for  patient  satisfaction 
(Westaway,  Rheeder,  Van  Zyl  &  Seager,  2003).  The  majority  of  literature  reviewed  agrees  that 
the  relationship  between  the  patient  and  prov  ider  has  an  impact  on  a  patient’s  satisfaction.  The 
provider-patient  relationship,  a  unique  challenge  in  itself,  is  considered  crucial  to  appropriate 
health  care  delivery  and  it  affected  by  culture  and  an  individual’s  background.  Just  as  a 
provider’s  background  influences  the  manner  in  which  they  diagnose  and  treat  a  patient,  a 
patient’s  background  (to  include  their  perceptions  and  expectations)  influences  their  actions  and 
behaviors  towards  their  provider  and  care  (Givaudan,  Pick.  DeVenguer  &  Xolocotzin,  2002). 
Patient  perceptions  influence  satisfaction.  “Patients  constantly  judge  the  motives  and 
competence  of  caregivers  through  their  interaction  with  them.  This  judgment  is  a  very  personal 
one,  based  on  perceptions  of  care  being  responsive  to  patients'  individual  needs,  rather  than  to 
any  universal  code  of  standards’’  (Wolosin,  2007,  p.  4).  Patient  satisfaction  is  higher  when 
health  education  and  discussion  with  provider  takes  place  during  a  patient’s  visit  (Robbins  et  al., 
1993).  Age  (older,  more  satisfied),  education  (more  education,  more  satisfied),  and  level  of 
health  and  illness  of  the  patient  (healthier  more  satisfied),  ethnicity  (minorities  less  satisfied),  the 
provider-patient  relationship  and  the  size  of  the  hospital  have  been  found  to  influence  a  patient’s 
satisfaction  (Cheng,  Yang  &  Chiang,  2003;  Wright,  et  ah,  2006).  A  study  by  Finstuen  and 
Mangelsdorff  also  noted  that  age  and  health  status  affected  levels  of  patient  satisfaction  for  care 
received  in  military  treatment  facilities  (Mangelsdorff  &  Finstuen.  2003).  Older  patients 
consistently  reported  higher  levels  of  satisfaction,  gender  did  not  influence  satisfaction,  however, 
ongoing  issues  with  communication  negatively  influences  satisfaction  levels  (Baker,  1993). 
Patient  satisfaction  studies  among  genders  has  produced  mixed  results,  and  depending  on  the 
report  you  view,  women  or  men  may  be  significantly  more  satisfied  or  that  there  may  be  no 
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significant  relationship  at  all  (Wright,  et  al..  2006).  As  seen  throughout  this  review,  a  majority  of 
literature  did  not  find  a  correlation  with  satisfaction  and  gender,  although  a  few  did.  For 
example,  Cho  (Cho,  Kim,  Chi  &  Kanho,  2007)  found  older,  female,  married  and  less-educated 
individuals  are  more  likely  to  be  satisfied.  A  Department  of  Veterans  Health  Administration 
study,  found  that  women  were  consistently  more  satisfied  w  ith  continuity  of  care  and 
prescription  services  and  less  satisfied  with  location  of  care,  than  men  (Wright,  S.,  Craig,  T., 
Campbell,  S.,  Schaefer,  M.  &  Humble,  C..  2006).  Early  studies  of  military  health  system 
(TRICARE)  beneficiaries  indicated  that  they  were  usually  satisfied  with  care,  though  higher 
levels  of  satisfaction  were  reported  for  those  receiving  care  away  from  the  military  treatment 
facility  (e.g.,  highest  satisfaction  in  retiree  personnel  and  lowest  in  active  duty  dependents) 
(Mangelsdorff,  1994).  White  ( 1 999)  provides  good  reason  to  not  group  scores  together  as  the 
APLSS  does,  as  it  discovered  patients  who  perceived  their  health  as  ‘very-good’  had  high  levels 
of  satisfaction  (‘neutral’  rating  had  the  lowest  levels);  grouping  or  generalizing  provides  room 
for  error.  Quintana  (Quintana  et  al..  2006)  found  men  with  high  levels  of  satisfaction  were 
influenced  by  comfort,  visiting  and  intimacy.  Providers  who  are  capable  of  identifying  and 
understanding  a  patient’s  feeling  were  shown  to  hav  e  higher  levels  of  satisfaction.  Patients 
whom  experience  a  smooth  continuity  of  care  have  high  levels  of  satisfaction  (Fitzpatrick  & 
Hopkins,  1993;  Bell  &  Krivich,  2000).  When  the  1HO  is  unable  to  meet  the  access  to  care 
standards  promised  to  its  beneficiaries  with  internal  resources,  the  I  HO  refers  its  beneficiaries  to 
one  of  24  MOU  HNH  to  ensure  its  promised  access  to  healthcare.  Access  to  care  has  shown  to 
be  an  influence!'  on  patient  satisfaction  (DuBovce,  2008). 

The  majority  of  HNH  doctors  can  communicate  in  English  (i.c.,  many  trained  in  the  U.S.), 
however  the  nursing  staff  may  not  be  able  to,  as  their  training  is  primarily  held  in  their  host 
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nation  and  does  not  center  on  English  (Allen  &  Little,  2007).  One  provider  and  medical 
professor  stated,  “to  provide  safe  and  quality  care,  a  physician  has  to  understand  the  patient”  and 
believes  language  barriers  lead  to  compromised  quality  of  care  and  preventable  medical  errors 
(Jordan,  2007,  p.  1).  Providers  and  patients  share  the  same  concern  with  language  barriers 
during  the  delivery  of  healthcare.  The  Calilornia  Academy  of  Family  Physicians  in  a  2003 
survey  found,  “over  half  of  all  providers  knew  of  one  or  more  occasions  in  which  the  delivery  of 
healthcare  was  compromised  by  language  barriers"  (Jordan,  2007).  A  successful  provider- 
patient  relationship  requires  communication  and  the  language  barrier  may  have  an  influence  on 
U.S.  beneficiary’s  satisfaction  during  their  healthcare  experience.  A  study  by  the  University  of 
California  Center  for  Health  Policy  Research  found  that,  “language  barriers  between  patients  and 
healthcare  providers  result  in  longer  hospital  stays,  more  medical  errors  and  lower  patient 
satisfaction"  (NgoMetzger  et  ah,  2007,  p.  1).  Jordan  went  further  to  state  such  barriers  cause 
unnecessary  tests,  delayed  diagnoses,  and  patients  not  properly  following  their  prescribed 
treatment  plans  (Jordan.  2007).  Access  to  English  speaking  staff  is  limited  and  is  sometimes 
unavailable  during  evening  hours.  U.S.  beneficiaries  are  concerned  with  whether  or  not 
provider-patient  dialog  is  translated  as  intended.  An  active-duty  (military)  family  member, 

Jamie  Yenco,  shared  her  experience  as  both  "nerve-racking  and  scary. .  .although  we  have 
interpreters,  you  don't  always  get  the  full  story”  (Allen,  2007b,  p.  2).  Technical  Sergeant  Ortiz, 
stationed  in  Japan,  stated,  “there  was  limited  interaction  with  the  technician  and  that  there  was  no 
play-by-play... or  conversation  beyond  the  initial  greeting"  (Allen,  2007b,  p.  4). 

Patient  satisfaction  is  an  outcome  of  quality  care,  not  vice-verse.  Data  from  National 
Committee  for  Quality  Assurance  (NCQA)  shows  that  access  is  a  critical  variable  of  influence 
for  patients’  satisfaction  (White,  1999).  Donabedian  views  satisfaction,  “as  a  judgment  on  the 
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quality  in  all  its  aspects,  but  particularly  as  concerns  the  interpersonal  process”  while  other 
experts.  Ware,  Synder,  Wright  and  Davis,  defined  satisfaction  as  a  set  of  components, 
“interpersonal  skills,  technical  quality,  accessibility,  cost,  efficacy,  continuity,  the  physical 
environment,  and  availability  of  resources”  (Westaway,  Rheeder,  Van  Zyl  &  Seager,  2003,  p. 
338).  Gerteis  (Gerteis,  Edgeman-Levitan,  Daley,  &  Delbanco,  1093)  believed  that  the  delivery  of 
healthcare  should  weigh  heavily  on  the  patient's  experience  of  illness  and  the  patient-provider 
relationship  should  be  one  of  collaboration  throughout  the  entire  patient-provider  encounter. 
Many  U.S.  beneficiaries  and  military  referring  providers  are  pleased  with  the  host  nation  care 
and  are  eagerly  willing  to  recommend  such  care  to  others.  Many  U.S.  beneficiaries  stated  that 
their  HNI  I  staff  was  very  attentive,  polite,  spoke  excellent  English  and  received  top-notch  care 
(Allen,  2007a).  A  provider  in  Japan,  LCDR  Ecker  stated,  “I  feel  very  confident  in  referring 
patients...  I  have  had  five  patients  treated  there...  they  have  one  of  the  top  five  endovascular 
surgeons  in  Japan”  (Allen  &  Little,  2007). 

Being  able  to  receive  care  locally  may  positively  influence  a  U.S.  beneficiary’s  satisfaction. 

A  DoD  employee  in  Japan.  Chip  Steitz,  stated,  “rather  than  fly  to  the  states  with  all  the  expenses, 
1  had  the  procedure  done  here  and  was  able  to  go  home  and  recuperate  with  my  wife  and  friends 
nearby...  I  probably  only  had  to  take  off  half  as  many  days  as  I  would  have  needed  had  I  gone 
back  to  the  States”  (Allen  &  Little,  2007,  p.  2).  Whether  or  not  IlNH  care  is  less  expensive  as 
compared  with  the  U.S.,  a  TRICARE  Standard  beneficiary  will  have  the  potentially  large 
financial  burden  to  pay  the  full  bill  at  the  time  of  discharge;  this  may  influence  a  U.S. 
beneficiary's  overall  satisfaction  with  their  experience.  For  example,  a  newspaper  reporter  died 
in  a  Japanese  hospital.  1  Iis  wife,  in  order  to  allow  her  to  take  the  body  and  properly  bury  it,  had 
to  take  out  a  loan  for  $20,000  to  repay  the  hospital  (Allen,  2007b). 
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Design  and  Potential  Biases 

The  patient  satisfaction  survey  was  designed  to  provide  a  reliable  and  valid  tool  for  leaders, 
providers,  planners  and  payers  to  gain  a  better  understanding  of  patients’  healthcare  experiences, 
current  or  potential  issues  and  challenges.  Most  importantly,  the  intent  of  this  data  is  to  provide 
a  continuous  means  to  evaluate  and  trend  patient  satisfaction  of  those  receiving  care  in  a  HNH 
facility;  when  required,  make  rapid,  effective  and  efficient  changes. 

Utilizing  a  survey  related  to  the  Army  Provider-Level  Satisfaction  Survey  (APLSS)  and  with 
a  similar  U.S.  beneficiary  population  helps  to  facilitate  comparing  data  both  internally  (IHO)  and 
with  APLSS  similar  data.  Prior  to  the  APLSS  full  deployment  covering  over  4,000  providers, 
there  were  several  pilot  tests  conducted  (by  the  third-party  program  administrator  Synovate)  with 
a  smaller  number  of  providers  geographically  dispersed  throughout  the  United  States  (Army 
Office  of  the  Surgeon  General,  2007).  The  APLSS  was  designed  for  continuous  tracking 
through  three  methods  of  data  collection  (telephonic,  mail  and  web-hased).  The  APLSS 
database  provides  Internet-based,  around-the-clock  access  and  reporting  capabilities,  allowing 
for  targeted  and  immediate  responsive  actions.  This  information  is  accessible  to  the  public 
through  the  TRICARE  Operations  Center  ( l'OC)  link  on  Office  of  the  Assistant  Secretary  of 
Defense  for  Health  Affairs  website  (http://www.ha.osd.mil/).  Making  information  available  to 
the  public  has  been  shown  to  influence  internal  quality  improvement  efforts  (Joshi,  Nash  & 
Ransom,  2005). 

The  Army  Surgeon  General  on  the  Army  Medical  Department  Command  Management 
System,  along  with  fiscal  year  2008  Business  Plan  guidance,  has  directed  a  two  percent  increase 
in  APLSS  question  7  (scores  of  4  and  5  combined),  patient’s  overall  satisfaction  with  provider, 
placing  the  satisfaction  target  at  95%.  along  with  achieving  high  marks  in  several  (satisfaction 
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related)  HED1S  measures  (Army  Office  of  the  Surgeon  General,  2007;  Army  Medical 
Department  CMS,  2008).  HEDIS  measures  primarily  focus  on  preventive  measures  (e.g., 
screening  of  cancers)  and  educating  patients  as  to  the  type  of  care  they  should  receive,  this 
allows  for  increased  dialog  between  the  provider  and  patient  (i.e.,  influential  in  patient 
satisfaction).  The  military  health  system  continues  to  outsource  the  APLSS,  as  many  healthcare 
organizations  believe  it  is  more  efficient  to  outsource  than  to  maintain  appropriate  staff  to 
conduct  surveys  (Bell  &  Krivich,  2000).  To  include  patient  demographics  in  the  survey  provides 
indications  about  potential  problem  areas  among  specific  groups  or  populations.  Joshi  (Joshi, 
Nash  &  Ransom.  2005)  indicates  that  young,  old  and  Medicaid  eligible  patients  are  more  prone 
to  not  complete  satisfaction  surveys.  It  is  also  stated  that  the  angriest  patients  are  the  least  likely 
to  complete  a  satisfaction  survey.  ‘"It  is  not  surprising  that  90%  or  more  of  patients  say  they  are 
satisfied,  because  the  language  of  satisfaction  is  too  restricted  to  convey  the  many  nuances  of 
attitude”  (Baker.  1993.  p.  58). 

The  MOU  HNH  survey  is  a  point-of-service  sun  ey.  meaning  is  it  given  to  the  patient  near 
time  of  discharge,  while  the  APLSS  is  primarily  a  mail-return  survey.  Point-of-service  surveys 
provide  rapid  feedback  (which  allows  the  staff  to  address  concerns  right  away),  by  design  are 
relatively  inexpensive  to  maintain,  and  provide  customizable  reports  at  any  time.  There  are  a 
few  downsides  to  the  point  of  survey  method:  is  it  a  continuous  system,  patients  may  not  feel 
completely  lucid  at  the  time  of  completion,  patients  may  feel  pressured,  patients  may  feel  that 
anonymity  is  missing,  no  means  to  control  the  process  (you  get  the  survey  back  when  you  get  it), 
potential  of  over-sampling  of  group,  and  keeping  the  information  private  is  always  a  concern 
(Bell  &  Krivich,  2000).  Survey  questions  should  address  quality,  access  and  interpersonal 
issues.  It  is  critical  to  include  an  overall,  “how  satisfied  were  you  with  your  provider”  and  “how 
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satisfied  were  you  with  the  facility  visit’".  This  allows  the  researcher  to  assess  overall 
performance  and  provides  a  single  metric  to  do  so.  Satisfaction  questions  should  have  a  scale  to 
answer.  White  noted  a  statement  from  A.C.  Myers  III  (CEO  and  President  of  the  healthcare 
survey  firm  Myers  Group),  "the  most  generally  used  and  accepted  scale  that  you'll  see  quoted  in 
the  literature  and  utilized  by  the  NCQA  is  the  five-point  scale...  advocating  a  five-point  scale 
ranging  from  poor  to  excellent.  Myers,  as  well  acknowledged  that,  in  general,  questionnaires  are 
designed  to  ask  just  about  every  question  possible,  but  typically  forget  to  ask  the  essential 
question,  “overall,  how  satisfied  are  you  with  your  physician”  (White,  1999,  p.  3).  Guadagnino 
(2003)  is  in  agreement  with  the  Myers,  confirming  that  it  is  extremely  common  for  surveys  to 
use  a  five-point  scale  ranging  from  very  poor  to  very  good.  Collecting  patient  demographics 
allows  identifying  trends  w  ithin  specific  groups  or  populations.  If  a  survey  is  perceived  as 
anonymous,  patients  are  more  prone  to  respond  truthfully  to  the  questions.  Mailing  surveys, 
rather  than  staff  handing  them  out,  is  recommended.  Patients  may  he  influenced  by  staff 
(increasing  potential  error/bias)  when  they  are  handed  to  them  (e.g.,  pressured  to  rapidly 
complete  them).  Short  surveys  are  also  recommended  as  long  surveys  may  lead  to  low 
motivation  for  patients  to  complete  the  surveys,  resulting  in  a  low  response  rate,  a  biased  sample 
and  potentially  missing  data  (Tso,  Ng  &  Chan.  2006).  Low  response  rates  may  introduce  a 
margin  of  error  and  a  35%  response  rate  is  typical  for  a  mailed  survey,  however  the  more 
responses  you  get  the  more  valid  and  reliable  your  results  are  likely  to  be  (White,  1999). 
According  to  Dr.  Glincr.  APLSS  Senior  Survey  Statistician,  a  20%  response  rate  is  typical; 
however  the  APLSS  strives  for  a  30%  response  rate  to  be  representative  of  the  population  (Army 
Office  of  the  Surgeon  General,  2007).  During  statistieal  analysis  it  is  recommended  not  to  lump 
responses  together,  especially  if  there  is  a  low  response  rate  as  it  increases  room  for  error  (White, 
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1 999).  The  APLSS  practices  this,  combining  scores  of  satisfied  and  completely  satisfied 
together  into  a  single  category  of  satisfied  (Army  Office  of  the  Surgeon  General,  2007).  This 
project  will  follow  the  same  technique  of  the  APLSS.  combining  scores  (Q10  and  Q1 1)  of  4  and 
5  to  show  satisfaction,  in  order  to  compare  results  uniformly.  One  of  the  main  concerns 
providers  continue  to  express  is  that  many  of  the  satisfaction  results  received  are  not  timely,  nor 
actionable  (an  inherent  bias).  Additionally,  providers  recommend  questions  be  more  objective. 
Instead  of  asking  how  satisfied  were  you  with  your  provider,  ask  something  more  actionable,  for 
example,  in  a  way  you  could  understand,  did  your  provider  explain  what  was  being  done  and 
why.  Surveys  which  are  conducted  in  the  same  manner  with  standardized  questions  may 
produce  more  reliable  and  valid  comparisons  (Joshi,  Nash  &  Ransom,  2005). 

Potential  Benefits  for  Having  High-Satisfaction 

There  is  a  great  deal  of  benefit  to  be  gained  from  having  accurate  and  timely  patient 
satisfaction  information,  aside  from  pro\  iding  leaders  the  ability  to  view  current  standings  across 
their  organization.  pro\  iders,  and  clinics.  Patient  satisfaction  results  may  provide  an  additional 
indicator  for  improvement  competition  and  bragging  rights.  Patient  satisfaction  results  have 
become  a  predictor  of  outcomes  (e.g.  patient  compliance,  regulatory  compliance,  market  share, 
patient  loyalty,  population  health  status  workload,  revenue).  Patient  satisfaction  feedback  (data) 
provides  valuable  information  that  may  improve  the  delivery  of  healthcare.  Satisfaction  data  can 
identify  areas  of  potential  trouble;  provide  critical  data  to  monitor  a  variety  of  medical  delivery 
models  and  to  make  appropriate  health  care  purchasing  decisions.  The  NCQA  requires  patient 
satisfaction  data  for  all  managed  care  organizations  applying  for  their  accreditation  (Kerr,  Hays, 
Mitchinson,  Lee  &  Siu.  1 999).  Managed  care  organizations  are  using  this  information  not  only 
as  a  measurement  of  performance  tool,  but  as  a  determinant  of  cost  (or  discounting)  of  current 
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and  future  contracts,  including  pay-for-performance  schedules.  Satisfied  patients  initiate  less 
litigation  (Guadagnino.  2003;  Wanless,  2007).  Tso  posited  that  an  organization  with  timely  and 
analyzed  patient  satisfaction  data  can  improve  care  and  services,  leading  to  better  patient  health 
(Tso.  Ng  &  Chan,  2006).  Poor  patient  satisfaction  can  turn  into  litigation.  In  his  article 
Guadagnino  (Guadagnino.  2003),  referred  to  a  June  2002  study  (by  Hickson.  Federspiel,  Pichert, 
et  al.,  2002),  in  the  Journal  of  the  American  Medical  Association  (volume  287,  issue  22),  which 
stated  that  lawsuits  were  considerably  influenced  by  the  number  of  patient  complaints.  “Past 
research  on  patient  satisfaction  has  found  a  linkage  between  satisfaction  and  hospital  utilization. 
While  a  majority  of  past  findings  supported  a  positive  relationship,  several  studies  presented 
evidence  for  an  inverse  relationship  between  satisfaction  and  frequency  of  outpatient  visits” 
(Cho,  Lee,  Kim,  Lee  &  Choi,  2004,  p.  14). 

An  additional  indicator  of  the  quality  of  an  industry  is  its  level  of  success  in  the  investment 
market  and  its  financial  viability.  The  healthcare  sector,  now  and  in  recent  years,  has  performed 
well  on  Wall  Street.  Bell  (Bell  &  Krivich,  2000)  found  that  the  strong  correlation  between  a 
hospital's  quality  score  and  its  operating  margin  and  that  nearly  25  percent  of  the  variation  in 
profitability  could  be  accounted  for  b  v  the  patients'  perceptions  of  quality.  Standard  and  Poor’s 
is  researching  methods  to  include:  patient  satisfaction  influenced,  quality  indicators  into  their 
bond  ratings  (White,  1999;  Bell  &  Krivich,  2000). 

Measuring  satisfaction  represents  care,  compassion  and  the  drive  to  improve  your  staff  and 
community.  Keeping  your  customer  satisfied  increases  loyalty  to  your  organization.  Retention 
is  substantially  cheaper  (5x)  than  recruiting  new  patients  (White,  1 999).  Patient  satisfaction  has 
been  linked  to  employee  satisfaction  and  retention,  competitive  market  strength,  profitability  and 
risk  management  (e.g..  decrease  in  litigation)  (Wolosin,  2001).  As  the  level  satisfaction 
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increases  among  patients,  so  in  turn  does  the  patients'  likelihood  to  comply  with  their  medical 
regimens  and  provide  critical  data  to  their  providers  (Fitzpatrick  &  Hopkins,  1993,  p.  4;  Ferris  L. 
E.,  1992;  Kerr,  Hays,  Mitchinson,  Lee  &  Siu,  1999).  Satisfaction  data  can  aid  in  staff  training, 
this  may  improve  the  provider-patient  relationship  as  patients  become  more  educated  (e.g., 
understanding  complication  rates,  report  cards,  clinical  quality)  providers  can  use  this  data  to 
identify  and  maintain  currency  of  patient  concerns  (Guadagnino,  2003).  “It  has  been  proposed 
that  the  effectiveness  of  health  care  is  determined,  to  some  degree,  by  satisfaction  with  the 
services  provided. ..a  satisfied  patient  is  more  likely  to  utilize  health  services,  comply  with 
medical  treatment  and  continue  with  the  health  care  provider”  (Westaway,  Rheeder,  Van  Zyl  & 
Seager,  2003,  p.  338).  A  patient  satisfaction  study  by  Tucker  (1998)  enforced  the  importance  of 
patient  satisfaction  studies.  Tucker  discussed  previous  study  findings  and  the  importance  they 
have  on  the  military,  for  example,  “that  military  readiness,  patient  compliance  with  prescribed 
regimens,  and  lower  overall  health  care  costs  are  all  correlated  with  patient  satisfaction”.  Tucker 
additionally  stated  that,  “health  care  ranks  within  the  five  most  important  quality-of-life  issues 
associated  with  morale,  which  is  a  short-term  determinate  of  readiness”  (Tucker,  1998,  p.  758). 

Healthcare  has  a  unique  challenge,  unlike  other  industries  they  have  to  first  take  care  of  the 
patients’  needs  versus  their  desires.  For  this  reason,  there  is  a  motivation  (by  all  parties)  for  a 
close  provider-patient  relationship.  Involving  patients  throughout  their  care,  increases 
satisfaction  and  meets  many  of  their  desires.  Patient  satisfaction  information  can  indicate  in  an 
organization,  what  works  well  (or  does  not),  organizations  can  then  allocate  resources 
appropriately  (perhaps  increasing  workload  and  decrease  staffing  issues)  (Wanless,  2007). 
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The  Department  of  Defense 

For  over  10  years,  the  Department  of  Defense  has  been  surveying  the  satisfaction  of  the 
TRICARE  program.  A  large  amount  of  resources  continues  to  be  committed  towards  this  DoD 
program.  This  program  obtains  critical  satisfaction  information  from  over  140  healthcare 
facilities  worldwide;  mailing  out  an  estimated  7.000  surveys  per  day  (United  States  Department 
of  Defense,  1997).  The  DoD  projects  military  health  expenditures  for  fiscal  year  2008  to  be  over 
40  billion  dollars  (Winkenwerder,  2007).  As  such,  the  American  people  expect:  “  a  fit,  healthy 
and  protected  force,  reduced  deaths,  injuries,  and  diseases  during  military  operations,  and 
superior  follow-up  care,  seamless  transition  with  the  Veterans  Administration,  satisfied 
beneficiaries,  creation  of  healthy  communities,  and  effective  management  of  healthcare  costs” 
(Winkenwerder,  2007,  p.  2).  Healthcare  leaders  in  the  military  are  shown  to  use  information 
from  patient  satisfaction  surveys  as  a  metric  of  provider  and  organizational  performance.  Having 
a  patient  satisfaction  feedback  tool,  as  the  APLSS,  allows  for  continuous  alignment  of  the 
changing  beneficiary  demographics  and  needs  of  the  military  health  system  (Army  Office  of  the 
Surgeon  General,  2007).  As  beneficiary  populations  are  becoming  critically  tied  to  funding 
(capitation  and  prospective  payment  system)  for  military  treatment  facilities,  maintaining  a 
satisfied  customer  population  becomes  all  that  more  important.  This  information  may 
additionally  provide  leverage  in  negotiations  with  network  providers  and  third  party  payers.  As 
TRICARE’s  Pacific  Regional  Office  (i.e.,  TRICARE  is  divided  into  regions  throughout  the 
world)  is  currently  working  on  a  Regional  Managed  Care  contract,  and  metrics  such  as  patient 
satisfaction  are  being  incorporated.  Gwen  Brown,  Deputy  Assistant  Secretary  of  Defense  for 
Health  Budgets  and  Program  in  a  1997  interview  stated  “we  want  to  know  how  people  view  their 
experience. .  .this  survey  lets  customers  know  we  really  care  about  them.  We  want  to  know  what 
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they  think  of  the  treatment  they  received,  their  provider,  and  the  facility. .  .we  need  this 
information  to  improve  the  system.  We  don't  want  to  bombard  our  customers  with  requests  for 
information,  so  we’ve  purposely  made  this  survey  short  and  specific.  We  can  benchmark 
satisfaction  levels  at  military  treatment  facilities  with  those  in  civilian  health  maintenance 
organizations”  (Gillert,  1997.  p.  1).  Although  the  military  's  workload  and  pace  has  been 
extremely  demanding,  with  limited  resources,  satisfaction  with  TRICARE  has  consistently 
improved  over  the  past  decade.  In  addition  to  “adding  performance-based  and  patient-centered 
care  initiatives  throughout  the  military  health  system,  we  added  financial  incentives  to  improve 
beneficiary  satisfaction  from  our  contract  partners  and  ensured  our  contractors  are  financially 
rewarded  for  care  delivered  in  the  private  sector”  (Winkenwerder.  2007,  p.  6).  “Consumers  are 
becoming  more  engaged  in  their  own  health  care  than  in  the  past... demanding  more  information 
and  choice  from  their  providers,  engaging  in  more  self-care  and  self-management  of  disease,  and 
showing  more  interest  in  participating  in  their  healthcare  decisions... the  unique  preferences, 
concerns  and  expectations  a  patient  brings  to  a  clinical  encounter... must  be  integrated  into 
clinical  decisions  to  serve  the  patient”  (Quadrennial  Defense  Review.  2006,  p.  32). 

Purpose  of  the  Project 

The  purpose  of  this  project  (using  a  reliable  and  valid  survey  instrument)  was  to  determine 
U.S.  beneficiary  satisfaction  (needs,  wants,  desires;  perceptions/expectations)  of  all  IHO  referred 
outpatient  care  received  at  HNH  (Korean)  and  if  overall  satisfaction  w  ith  provider  meets  the 
fiscal  year  2007  Department  of  Defense/Military  Health  System’s  goal  of  95%  satisfied  (with 
scores  4  and  5  combined).  This  project  additionally  looked  at  the  relationship  between  U.S. 
beneficiary  patient  satisfaction  and  facility  \  isit  (if  it  too  met  a  95%  satisfaction  level), 
beneficiary  category,  command  sponsorship  and  gender.  Retaining  or  increasing  the  U.S. 
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beneficiary  market  share  is  not  a  primary  concern  in  Korea,  as  the  healthcare  delivery  system  is 
designed  around  a  closed  network  with  limited  resources.  Patient  satisfaction  information 
provides  leadership  with  a  patient  perspective  of  care  received.  This  information  is  key  in 
maintaining  a  healthcare  system  (i.e.,  in  this  unique  setting-Korca)  which  provides  access  and 
quality  to  its  beneficiary  population.  1  he  survey’s  data  provides  an  understanding  of  how 
satisfied  or  potentially,  dissatisfied  patients  are.  Leadership  and  providers  can  incorporate  this 
information  into  future  planning  (changes),  training  and  evaluations.  This  information  has 
identified  both  the  good  and  bad  (successes  and  failures)  within  the  current  system.  Timeliness 
of  survey  feedback  to  IHO  and  HNH  leadership  is  critical,  this  may  allow  for  rapid  changes  to  be 
made  with  the  system  when  necessary.  Reacting  to  feedback  in  a  timely  manner  displays  to  the 
beneficiary  population  that  the  leadership  not  only  reviews  and  is  concerned  about  beneficiary 
feedback,  it  reacts.  This  information  provides  a  check/balance  on  what  (and  how  many)  services 
are  referred  out  and  to  which  facility.  Results  allow  leadership  to  view  and  compare  each  Host 
Nation  Facility  separately.  TRICARE  Management  Activity  is  developing  a  managed  care 
contract  to  cover  the  Korean  Peninsula,  scheduled  to  begin  in  2009.  It  is  assumed  that  the 
managed  care  contract  will  initially  build  the  network  utilizing  the  current  MOU  HNH  and  will 
continue  a  modified  version  of  the  current  agreements.  Assessment  tools,  to  include  patient 
satisfaction  surveys,  will  become  a  requirement  for  the  managed  care  contract  and  contractor. 

The  dependent  variables  are:  (yi/Q10)  everything  considered,  how  satisfied  were  you  with 
the  facility  during  your  visit;  (y2/Ql  1)  overall,  how  satisfied  do  you  feel  about  your  visit  with 
your  provider;  (y3/Ql)  your  provider  listened  to  you  carefully  about  your  concerns;  (y4/Q6) 
courtesy  and  helpfulness  of  the  staff  during  this  visit;  and  (ys/Q7)  the  coordination  among  all  the 
people  who  cared  for  you  during  this  visit.  1  he  independent  variables  are  (the  demographics): 
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(xi)  gender;  (X2)  beneficiary  category;  (X3)  command  sponsorship.  The  following  hypotheses 
will  be  tested: 

1.  Hal:  The  patients'  overall  satisfaction  during  ‘visit  with  their  (HNH)  provider’  (Q1 1) 
meets  FY07  DoD/MHS  goal  of  95%  satisfied. 

2.  Ha2:  The  patients'  overall  satisfaction  with  (HNH)  ‘facility  visit'  (Q10)  is  at  least 
95%  satisfied. 

METHODS  and  PROCEDURES 

This  project  identifies  and  addresses  the  MOU  HNH  referred  US  outpatient  beneficiary 
population  and  their  satisfaction  levels  with  MOU  HNHs.  The  means  and  methodology  used  to 
collect  and  analyze  this  data,  including  additional  categorical  and  quantitative  data,  was  derived 
primarily  from  the  MOU  HNH  (outpatient)  survey,  local  and  military  databases  (including  the 
IHO’s  TRICARE  Office  HNH  database),  the  Military  Health  System’s  Management  Analysis 
and  Reporting  Tool  (M2),  the  Army  Medical  Department  Command  Management  System,  ICE, 
and  the  APTSS  online  reporting  system.  The  MOU  HNH  survey  additionally  collects 
demographic  data  (e.g.,  gender,  beneficiary  category  and  command  sponsorship)  of  U.S. 
beneficiaries  in  order  to  identify  potential  satisfaction  indicators.  The  satisfaction  information 
was  compared  (using  inferential  statistics)  \\  ith  data  in  the  APLSS  system  to  identify  how 
satisfaction  may  vary  among  the  MOU  HNHs  and  the  IHO.  MOU  HNH  survey  questions  10  and 
1 1  were  compared  against  the  corresponding  APLSS  questions  21  and  7  providing  IHO 
leadership  an  additional  quality  indicator  tool  for  planning.  Additionally,  correlations  on  the 
MOU  HNH  questions  were  conducted  (using  raw  data).  This  information  provides  additional 
insight  to  allow  leadership,  both  at  the  IHO  and  HNH,  to  make  appropriate  reactionary  and 
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proactive  decisions  relating  to  meeting  the  needs  and  expectations  of  their  beneficiary 
populations. 

Type  of  Analysis 

The  Statistical  Package  for  Social  Science  (SPSS)  version  16  for  MAC,  XLSTAT  2008  and 
Microsoft  Excel  Statistical  Add-On  for  statistical  (quantitative)  analysis  was  utilized  for  this 
project.  A  descriptive  summary  of  U.S.  beneficiaries  (in  the  ROK  whom  are  IHO  beneficiaries 
referred  as  an  outpatient)  satisfaction  along  with  correlations  solely  to  test  the  hypotheses  was 
conducted.  Descriptive  statistics  for  the  MOU  HNH  survey  are  seen  Table  1  below.  To  test  the 
hypotheses,  Q10  and  Q1 1  (the  dependent  variables)  were  individually  totaled  (combining  Likert 
Scale  answers  of  4  and  5)  with  descriptive  statistics  (means  and  standard  deviations)  performed 
to  determine  the  overall  percentage  satisfied  for  each  question.  Satisfaction  is  defined  as  a 
response  of  “somewhat  satisfied  or  completely  satisfied”  on  Questions  10  and  1 1  of  MOU  HNH 
Satisfaction  Survey  (5  point  Likert  scale).  Collection,  statistical  analysis,  and  reporting  of  this 
data  is  now  on  an  on-going  basis  and  continually  is  aligned  (and  adjusted  as  required)  with  the 
needs  of  the  organizational  leadership.  Inferential  statistics  (ANOVA,  Chi-Square,  Pearson’s 
Correlation)  were  completed  to  determine  if  statistical  significance  existed. 
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Table  1 


Descriptive  Stalistics  frum  MOU  HNH  Survey  (September-Novemher  2007  data) 


n 

MIN 

MAX 

Mean 

SD 

Gender  (males-53;  females-61) 

114 

0 

1 

.47 

.50 

Command  Sponsored  (yes-95;  no- 19) 

114 

0 

1 

.83 

.37 

Ql .  Your  provider  listened  to  you  carefully 
about  your  concerns  and  questions 

114 

0 

l 

.90 

.31 

Q6.  Courtesy  and  helpfulness  of  the  staff 
during  this  visit 

114 

0 

1 

.85 

.36 

Q7.  The  coordination  among  all  the  people 
who  cared  for  you  during  this  visit 

114 

0 

1 

.86 

.35 

Q10.  Everything  considered,  how  satisfied 
were  you  with  the  facility  during  your  visit 

1 14 

0 

1 

.92 

.28 

Ql  1.  Overall,  how  satisfied  do  you  feel 
about  your  visit  with  your  provider 

114 

0 

1 

.93 

.27 

Note.  Gender  (1  =  male;  0  =  female);  Command  Sponsored  (1  =  yes;  0  =  no);  Ql,  Q6,  Q7,  Q10 
and  Ql  1  (1  =  satisfied  with  scores  4  and  5  combined  from  Likert  scale;  0  =  all  other  scores). 

Validity  and  Reliability 

The  MOU  HNH  survey  was  designed  and  aligned  in  accordance  with  the  Army  Provider 
Level  Satisfaction  Survey  in  order  to  increase  validity  and  reliability.  APLSS  mails  out  an 
average  of  7,000  surveys  per  day  with  an  average  response  rate  of  39%  (Army  Office  of  the 
Surgeon  General,  2007).  The  MOU  HNH  survey  questions  being  derived  from  the  extensively 
tested  and  utilized  Army  Office  of  the  Surgeons'  General  APLSS  increases  the  validity  and 
reliability  along  with  the  capability  to  compare  data  (within  the  APLSS  system).  As  White 
(1999)  stated,  the  more  responses  you  get  the  more  valid  and  reliable  your  results  are  likely  to 
be;  this  adds  to  the  reliability  and  validity  of  the  APLSS  by  its  large  number  of  responses.  The 
MOU  HNH  survey  questions  are  aligned  with  the  APLSS  questions  (for  increased  reliability  and 
validity)  and  surveys  the  same  U.S.  beneficiary  population.  Utilizing  a  standardized  patient 
satisfaction  monitoring  and  reporting  tool  may  allow  healthcare  leadership;  opportunities  to 
compare  among  similar  facilities,  more  effective  quality  indicators,  avoid  and  remedy  potential 


Beneficiary  Satisfaction  In  Korea  41 


healthcare  delivery  obstacles  and  risks,  and  if  required,  to  react  accordingly  and  in  a  timely 
manner.  Standardize  tools  are  critical,  just  as  Davies  acknowledged  following  the 
implementation  of  the  NCQA  member  health  survey,  “for  plans,  payers,  and  the  public,  this  long 
awaited  standardization  provides  opportunities  for  benchmarking,  comparison  and  better- 
informed  decision-making”  (Davies,  Ware  &  Kosinski,  1995).  Synovate,  the  company 
contracted  to  design  the  APLSS,  provided  an  information  paper  regarding  the  development  and 
validity  (to  include  validity  evaluation  indicators  used)  of  the  survey  (see  Appendix  G). 
Synovate,  one  the  largest  custom  research  firms  in  the  world,  is  DoD  Information  Technology 
Security  Certification  and  Accreditation  Process  compliant  and  National  Committee  on  Quality 
Assurance  certified  (Health  Net,  2006).  Synovate  conducted  several  analyses  throughout  each 
stage  in  the  APLSS  lifecycle  development  and  implementation.  Synovate,  in  addition,  leveraged 
results  from  existing  healthcare  surveys  (e.g..  CAHPS,  Kaiser  Permanente  questionnaires),  and 
Army  Medical  Department  staff  input  regarding  the  survey  questions  soundness  (Synovate 
Healthcare,  2007). 

Along  with  the  descriptive  statistics  table,  this  project  includes  frequency  distributions  for  all 
variables  measured.  This  project  assessed  the  internal  validity  (it  appropriately  addressed  the 
intended  hypothesis-  y  =  fix)),  the  external  validity  (the  results  can  be  applied  to  related 
populations  and  groups)  and  face  validity  (the  data  appeared  valid  and  did  measure  what  it  was 
designed  for).  During  the  hypothesis  testing,  inferential  statistics  (e.g.,  ANOVA,  Chi-Square, 
and  Pearson’s  correlation)  were  conducted  via  SPSS  and  XLSTAT  on  the  dependent  variables: 
(yi/Q10)  everything  considered,  how  satisfied  were  you  with  the  facility  during  your  visit; 

(yj/Ql  1)  overall,  how  satisfied  do  you  feel  about  your  visit  with  your  provider  and  independent 
variables:  (xi)  gender;  (X2)  beneficiary  category;  (X3)  command  sponsorship.  To  ensure  the 
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survey  remained  straightforward  and  questions  meant  the  same  to  all  potential  readers  of  the 
survey,  I  additionally  conducted  a  Flesch-Kincaid  index  for  reading  ease  (i.e..  length  of 
sentences  and  number  of  syllables  per  word),  which  was  57.4  and  grade  level  of  8.6. 

Ethical  Considerations 

The  Tripler  Army  Medical  Center  (TAMC)  partners  with  the  18tn  Medical  Command  as  their 
Institutional  Review  Board  (IRB)  (Leighnor  &  Person.  2007)  of  record.  The  TAMC  IRB  was 
contacted  and  was  provided  an  overview  of  this  project  along  with  a  sample  of  the  survey.  The 
project  and  survey  were  not  subject  to  review'-and-approval  by  the  TAMC  IRB  (see  Appendix  I) 
as  this  project  was  determined  to  place  emphasis  on  internal  purposes  and  improvement 
(Leighnor  &  Person,  2007).  “Military  treatment  facilities  may  conduct  local  level  surveys  of 
beneficiaries  and  assigned  personnel  to  address  the  need  for  more  specific  information  on  clinic 
operations  and  services  within  that  specific  military  treatment  facility. ..it  should  primarily  be 
administered  on  site  and  the  results  published  and  utilized  at  the  local  level.  Surveys  of  this 
nature  will  not  require  Health  Affair  or  TRICARE  Management  Activity  level  review  and 
approval'’  (Sears,  2000,  p.  2).  This  surv  ey  protects  the  anonymity  of  its  participants  as  the 
survey  does  not  include  a  means  to  determine  who  completed  the  survey  nor  can  it  determine 
specific  patient  data. 

Project  Design 

This  project  uses  inferential  statistics  to  identify  and  determine  if  any  quantitative  differences 
exist  between  them  with  overall  satisfaction  with  the  provider  and  facility,  after  an  outpatient 
visits  a  HNH  within  the  Republic  of  Korea  using  sample  size,  means  and  standard  deviations 
(see  Table  2  for  coding).  As  the  literature  review  indicated,  one  of  the  largest  influences  to 
patient  satisfaction  is  the  provider-patient  interaction.  The  IHO  continues  to  pursue  methods  to 
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increase  its  communications  and  coordination’s  among  patients,  staff  and  agencies.  For  these 
reasons,  this  project  focused  primarily  on  the  three  demographic  fields  (gender,  beneficiary 
category,  command  sponsorship),  Q10  and  Q 1 1  of  the  MOU  HNH  survey  and  potential 
correlations  amongst  survey  questions  themselves.  All  data  was  derived  from  the  TRICARE 
U.S.  beneficiary  referred  to  a  MOU  IfNH  as  an  outpatient  population.  A  beneficiary  is 
considered  an  outpatient  when  he  or  she  returns  home  the  same  day  as  the  medical  treatment 
facility  visit  and  or  procedure. 

Data  Collection 

Survey  data  collection  and  analysis,  for  this  project,  utilized  available  data  from  September 
through  November  (three  month  period)  2007.  The  period  represents  a  period  after  large 
personnel  transitions  and  prior  to  the  holiday  (exodus)  period.  The  data  was  entered  into  the 
database  for  statistical  analysis  as  it  was  received  by  the  IHO’s  TRICARE  Office.  This 
continuous  collection  and  analysis  (see  Figure  3)  ensures  leaders  have  an  ongoing  picture  (not  a 
semi-annual  or  annual  picture)  enabling  them  to  react  rapidly  and  intervene  when  it  is  needed; 
data  becomes  history  rapidly.  According  to  the  II 10' s  TRICARE  Service  Center’s  local 
database  and  as  seen  in  Appendix  A,  12  of  the  24  MOU  UNI  Is  are  in  the  heavily  U.S. 
beneficiary  populated  Seoul  area.  According  to  the  TRICARE  Service  Center's  database,  there 
was  an  estimated  6,000  total  referrals  in  calendar  year  2007  throughout  the  Korean  Peninsula  for 
US  beneficiaries.  In  order  to  establish  an  equally  distributed  sample  within  the  ROK,  three 
hospitals  geographically  separated  throughout  the  ROK  were  chosen.  These  three  hospitals, 
Samsung,  Dankook  and  Dongsan  represent  nearly  half  of  all  calendar  year  2007  U.S.  beneficiary 
outpatient  referrals  within  the  ROK.  For  this  reason  and  having  the  ability  to  control  (via  the 
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TRICARE  Service  Center  staff)  the  survey  distribution  and  collection,  this  project  used  these 


three  hospitals  as  its  U.S.  beneficiary  outpatient  sample  (n)  of  the  ROK  population  (N). 


Action  taken  to 
improve 
care  for 
future  patients 


Patient 

Satisfaction 

Survey 

Collected 


Data  Coded/ 
Entered  into 
SPSS 


Figure  3.  Conceptual  model  of  IHO  survey  data  collection  and  analysis. 


Survey  Instrument 

Although  the  APLSS  primarily  utilizes  the  mail  survey  method  (with  an  Internet  response 
option),  the  MOU  HNH  survey  uses  a  point-  of-service  survey  due  to  resource  constraints.  As  of 
20  December  2007.  the  MOIJ  HNH  survey  has  a  link  to  an  Internet  response  option  through  the 
ICE  system.  To  minimize  variation  from  the  intended  APLSS  data,  each  item  was  scored  on  a 
five-point  (five  possible  responses)  Likert  scale  (similar  to  the  APLSS)  ranging  from  one 
(completely  dissatisfied)  to  five  (completely  satisfied).  In  a  patient  satisfaction  questionnaire 
study  by  Mangelsdorff,  the  use  of  the  “Likert  scale  format  allowed  for  greater  discrimination  of 
the  intensity  of  a  respondent’s  belief  regarding  an  issue...  was  a  consistently  more  reliable 
method  to  produce  scores  than  using  other  methods  (e.g.,  Thurstone  method)’’  (Mangelsdorff, 
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1979.  p.  86).  Survey  seores  of  4  and  5  (somewhat  satisfied  or  completely  satisfied)  were 
combined  to  determine  the  percentage  of  overall  satisfaction  for  patients’  responses  to  Q10  and 
Q1 1  on  the  MOU  HNH  Survey.  The  MOU  HNH  survey,  as  seen  in  Appendix  C,  uses  a  Likert 
Seale  (1-5)  for  the  following  (APLSS  matching)  questions: 

Q1 .  Your  provider  listened  to  you  carefully  about  your  concerns  and  questions. 

Q2.  Your  provider  understood  your  problem  or  condition. 

Q3.  Your  provider  treated  you  with  courtesy  and  respect. 

Q4.  Your  pro\  ider  explained  what  was  being  done  and  why. 

Q5.  The  amount  of  time  you  waited  at  the  facility  to  see  the  healthcare  provider. 

Q6.  Courtesy  and  helpfulness  of  the  staff  during  this  visit. 

Q7.  The  coordination  among  all  the  people  who  cared  for  you  during  this  visit. 

Q8.  The  cleanliness  of  the  facility  you  visited. 

Q9.  The  eomfort  of  the  facility  you  visited. 

Q10.  Everything  considered,  how  satisfied  were  you  with  the  facility  during  your  visit? 

Q1 1.  Overall,  how  satisfied  do  you  feel  about  your  visit  with  your  provider? 

Surveying  Proeess 

For  those  U.S.  beneficiaries  utilizing  the  MOU-HNH  free  shuttle  system,  the  MOU  HNH 
survey  was  distributed  by  IHO  staff  upon  beneficiary  cheek-in  at  the  TRICARE  Service  Center. 
All  other  U.S.  beneficiaries  throughout  the  ROK  that  do  not  process  through  the  TRICARE 
Service  Center  are  provided  the  MOU  HNH  survey  at  the  point-of-serviee  by  International 
Health  Clinie  staff  at  the  MOU  HNH.  The  survey  was  either  eolleeted  upon  return  to  the 
TRICARE  Sen  ice  Center  by  the  TRICARE  staff  or  the  survey,  which  is  pre-printed  and 
stamped,  was  mailed  in  (at  no  cost)  direetly  to  the  TRICARE  Serviee  Center.  All  patients  had  an 
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equal  opportunity  to  respond  (and  to  do  so  anonymously).  Due  to  the  geographic  dispersion  of 
medical  facilities,  communications  between  the  facilities,  drivers,  and  patients  continues  to  prove 
a  rather  large  obstacle  in  controlling  the  distribution  and  collection  of  surveys  outside  of  the 
Seoul-Yongsan  area,  where  the  TRICARE  Service  Center  is  located.  For  this  reason,  it  is 
assumed  that  a  portion  of  the  51 1  U.S  beneficiary  outpatients  referred  (during  this  timeframe)  to 
a  HNH  have  not  received  the  opportunity  to  provide  feedback  via  the  MOU  HNH  survey  (a 
project  limitation).  The  ICE  online  survey  option  (with  questions  aligned  with  the  hardcopy 
MOU  HNH  Survey),  which  was  introduced  in  December  of  2007,  which  is  now  monitored 
closely  by  the  TRICARE  Service  Center  and  the  Patient  Advocate  to  determine  its  effectiveness 
for  the  beneficiaries  and  the  organizational  leadership  (e.g.,  response  rates  and  reporting 
formats).  Additional  process  improvement  options  are  under  discussion  with  the  TRICARE 
Service  Center  leadership  (e.g.,  incorporate  into  patient  transportation  briefings,  placing  a 
TRICARE  representative  in  each  of  the  four  geographic  areas,  and  training  at  the  MOU 
facilities). 

Survey  Return  Rate 

The  response  rate  was  calculated  by  dividing  the  number  of  returned  surveys  (1 14)  by  the 
number  of  U.S.  outpatient  beneficiaries  referred  (51 1)  to  MOU  facilities.  A  limitation  to  this 
project,  as  seen  later  in  the  Discussion  section  of  this  project,  refers  to  the  inability  of  the 
TRICARE  Service  Center  to  ensure  100%  of  referred  outpatients  received  a  survey  to  complete. 
The  goal  for  this  project  was  a  response  rate  of  25%,  however  only  a  21%  response  rate  (or 
approximately  38  per  month)  was  achieved  during  the  timeframe.  This  project  does  not  compare 
data  and/or  results  with  historical  data  and  time  periods  due  to  the  limited  amount  of  historical 


data. 
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Data  Coding 

The  software  package,  Statistical  Package  for  Social  Sciences  (SPSS)  16,  XLSTAT  2008  and 
Microsoft  Excel  Statistical  Add-On.  which  were  utilized  in  this  project,  required  data  (variables) 
to  be  coded  for  proper  statistical  analysis.  Table  2  describes  the  coding  utilized  for  SPSS  in  this 
project. 


Table  2 

Data  Coding  for  Statistical  Package  for  Social  Sciences  (SPSS) 

Variable _ Description _ SPSS  Code _ Data  Source 

Dependent  Variables 

Patient-Provider  Satisfaction  (scores  of  4  and  5  are  combined) 

Patient-Facility  Visit  Satisfaction  (scores  of  4  and  5  are  combined) 

Independent  Variables 


Gender 

Male 

1 

HNH  Survey 

Female 

0 

HNH  Survey 

BENCAT 

AD 

1 

HNH  Survey 

ADFM 

2 

HNH  Survey 

RET 

o 

HNH  Survey 

RETFM 

4 

HNH  Survey 

Other 

5 

HNH  Survey 

Command  Sponsored  Yes 

1 

HNH  Survey 

No 

0 

HNH  Survey 

Note.  The  beneficiary  category  "Other”  includes:  Department  of  State  and  Embassy  personnel, 
Korean  service  employees  and  Department  of  Defense  School  System  teachers. 

RESULTS 

A  sample  of  (n=l  14)  US  beneficiary  outpatients  referred  to  one  of  three  MOU  HNH  in  the 
ROK  were  surveyed  to  determine  whether  or  not  overall  satisfaction  with  provider  and  visit 
reached  the  DoD/MHS  95%  target.  During  the  months  of  September,  October  and  November  of 
2007  from  Dankook  Medical  Center,  Dongsan  University  Medical  Center  and  Samsung  Medical 
Center,  the  1110  (peninsula-wide)  received  1 14  surveys  (approximately  1 1%  of  all  ROK 
referrals).  This  project  resulted  in  a  21%  response  rate  over  the  period,  4%  less  than  originally 
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anticipated.  The  MOU  HNH  survey  results  did  not  support  either  alternate  hypothesis.  Overall 
satisfaction  with  provider  and  facility  visit  did  not  reach  the  DoD/MHS  target  of  95%. 
Descriptive  statistics  revealed  (see  Table  1)  the  following  overall  satisfaction  rates:  Ql-  90%, 
Q6-  85%,  Q7-  86%,  Q10  (Hal)-  92%,  and  Ql  1  (Ha2)-  93%.  Scores  of  four  and  five  are  grouped 
together  for  overall  satisfaction  percentages.  Frequency  distribution  tables  for  all  variables  can 
be  seen  in  Appendix  H.  As  seen  in  Table  3  and  Figure  4  below,  the  mean  patient  satisfaction  for 
the  MOU  HNH  was  higher  than  the  121 st  Combat  Support  Hospital  for  both  Q 1 0  and  Q 1 1 , 
however  only  higher  on  Ql  0  when  compared  to  the  Medical  Activity  in  Japan,  Camp  Zama  (the 
only  other  Army  Medical  Activity  in  the  region).  Chi-Square  results  indicated  no  significance 
for  Q10  (x2  =  2.01;  p  —  .37;  df=  2),  however  did  indicate  a  statistically  significant  result  for  Ql  1 
(x2=  13.3; p  =  .001;  df=  2). 


Table  3 

Satisfaction  Percentages. 

Organization 

Overall  Facility  Visit 

Overall  Provider 

Satisfaction  (Q10) 

Satisfaction  (Ql  1) 

MOU  Host  Nation  Hospitals 

92% 

93  % 

12U1  Combat  Support  Hospital 

85  % 

88% 

Camp  Zama  Medical  Activity 

90% 

95  % 

Note.  Data  collection  period  for  MOU  Host  Nation  Hospitals  1  September  through  31  November 
2007.  Due  to  pre-defined  timeframes  on  the  Army  Provider-Level  Satisfaction  Survey  (APLSS) 
online  reporting  tool,  data  collection  period  for  12L1  Combat  Support  Hospital  and  Camp  Zama 
Medical  Activity  27  August  through  16  December  2007. 
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Satisfaction  Comparison 


Survey  Questions 


Figure  4.  Overall  Visit  (QIC)  Satisfaction  (Likert  scale  scores  of  4  and  5  combined);  Overall 
Provider  (Q 11)  Satisfaction  (Likert  scale  scores  of  4  and  5  combined)  (Army  Office  of  the 
Surgeon  General,  2007;  HNH  Survey  Results,  2007).  Note.  121st  and  Camp  Zama  Medical 
Activity  timeframes  are  from  August  27  through  1 6  December:  MOU  HNH  includes  September 
through  November  2007. 


Descriptive  statistics  additionally  revealed  patient  satisfaction  means  for  gender,  beneficiary 
category  and  command  sponsorship  in  relationship  to  each  survey  question.  Table  4  and  Figures 
5,  6  and  7.  provide  graphical  representation  of  this  data.  Males  had  a  10%  higher  level  of  overall 
satisfaction  when  compared  to  females  for  all  survey  questions  analyzed  (mean  of  means); 
command  sponsorship  results  indicated  a  small  mean  difference  of  2  %;  active  duty  and  active 
duty  family  members  (represented  just  over  92  %  of  all  respondents)  resulted  in  a  2  %  mean 
difference,  higher  in  active  duty;  retiree  and  retiree  family  members  represented  4  %  of  all 
respondents  with  data  indicating  100  %  satisfaction;  other  beneficiaries  (Department  of  State  and 
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Embassy  personnel.  Korean  service  employees  and  Department  of  Defense  School  System 
teachers)  represented  4  %  of  all  respondents  and  data  showed  an  overall  satisfaction  of  84  %. 


Table  5 


Demographig  Data  Compared^  to  HNH  Survey  Questions  (Sept ember -November  2007) 


Survey  Male 
Question 

Female 

AD 

ADFM 

RET  RETFM 

Other 

CMD 

Sponsored 

Not  CMD  Overall 
Sponsored  Satisfied 

Q1  94.3 

85.2 

88.4 

91.7 

100 

100 

80 

88.4 

94.7 

89.5 

Q6  94.3 

77.0 

88.4 

80.6 

100 

100 

60 

85.3 

84.2 

85.1 

Q7  94.3 

78.7 

87.0 

83.3 

100 

100 

80 

86.3 

84.2 

86.0 

Q10  94.0 

90.2 

91.0 

92.0 

100 

100 

100 

93.0 

89.0 

92.1 

Qll  94.0 

88.5 

87.0 

97.0 

100 

100 

100 

89.0 

100 

92.9 

Note.  Satisfaction  data  based  on  Likert  scale  scores  of  4  and  5  combined  (HNH  Survey  Results, 


2007)  and  displayed  in  percentages. 


Satisfaction  Based  on  Gender 


Figure  5.  Overall  Satisfaction  based  on  Gender  with  Likert  scale  scores  of  4  and  5  combined 
(HNH  Survey  Results,  2007).  Note.  MOU  HNH  includes  data  from  September  through 


November  2007;  n  =  53  males  and  61  females. 
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Satisfaction  Based  on  Command  Sponsorship 


Survey  Question 


Figure  6.  Overall  Satisfaction  based  on  Command  Sponsorship  with  Likert  scale  scores  of  4  and 
5  combined  (HNH  Survey  Results,  2007).  Note.  MOU  HNH  includes  data  from  September 
through  November  2007;  n  =  95  command  sponsored  and  19  not  command  sponsored. 
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Satisfaction  Based  on  Beneficiary  Category 
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Figure  7.  Overall  Satisfaction  based  on  Beneficiary  Category  with  Likert  scale  scores  of  4  and  5 
combined  (HNH  Survey  Results,  2007).  Note.  MOU  HNI1  includes  data  from  September 
through  November  2007;  n  =  60  Active  Duty,  36  Active  Duty  Family  Members,  2  Retirees,  2 
Retiree  Family  Members  and  5  Others  (Department  of  State  and  Embassy  personnel,  Korean 
service  employees  and  Department  of  Defense  School  System  teachers). 


Beneficiary  Satisfaction  In  Korea  53 


The  following  inferential  statistical  tests  were  completed:  ANOVA  (beneficiary  category)  and 
Chi-Square  (gender  and  command  sponsorship)  to  determine  if  statistical  significance  existed. 
The  ANOVA  conducted  with  Q10  and  beneficiary  category  resulted  in:  F(4,109)  =  .728  with  a 
p-value  of  .574,  indicating  no  significance.  When  the  ANOVA  was  conducted  with  Q1 1,  there 
were  similar  results:  F( 4,  109)  =  .491  with  p-value  of  .742.  Chi-Square  results,  also  indicating 
no  significance,  were  as  follows:  QlO/Gender  (x  =  .68; p  =  .5;  df=  1);  Q1 1/Gender  (x  =  1.19; 
p  -  .5;  df=  1);  Q1 0/Command  Sponsorship  (x2  =  .22;  p  =  .75;  df—  1);  Q1 1/Command 
Sponsorship  (x2  =  2.19;/?  =  .25;  df=  1). 

Correlations,  between  patient  satisfaction  survey  responses  wrere  conducted  and  analyzed  for 
significance  using  raw  response  data  as  seen  in  Table  5.  These  correlation  coefficients  were 
additionally  squared  (R2)  to  determine  the  shared  variance  in  the  data  that  can  be  accounted  for 
and  the  closeness  of  the  relationships  between  the  responses  (“1”  being  a  perfect  relationship). 
Table  5  displays  several  significant  correlations:  Q1  and  Q6  with  an  R  of  .159;  Q1  and  Q7  with 
an  R‘  of  .249;  Q1  and  Q10  with  an  R^  of  .399;  Q1  and  Q1 1  with  an  R2  of  .384;  Q6  and  Q7  with 
an  R2  of  .710;  Q6  and  Q10  with  an  R  of  .403;  Q6  and  Q1 1  with  an  R2  of  .348;  Q7  and  Q10  writh 
an  R2 of  .515;  Q7  and  Q1 1  with  an  R2  of  .500;  Q10  and  Q1 1  with  an  R2  of  .678. 
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Table  6 

Correlations  Between  Survey  Responses. 

Ql 

Q6 

Q7 

Q10 

Qll 

Ql 

Pearson  Correlation 

l 

Q6 

Pearson  Correlation 

.398** 

1 

Q7 

Pearson  Correlation 

499** 

.843** 

1 

Q10 

Pearson  Correlation 

.632** 

.635** 

7  J7*  + 

1 

Qll 

Pearson  Correlation 

.619** 

.590** 

.707** 

.823** 

1 

Note.  Raw  data  was  used  for  these  correlations;  scores  of  4  and  5  were  not  combined. 

**  Correlation  is  significant  at  the  0.05  alpha  level  (/K.0001). 

DISCUSSION 

The  purpose  of  this  project  was  to  determine  whether  or  not  US  beneficiaries  are  satisfied  (in 
relation  to  the  DoD/MIIS  95%  target)  with  the  care  that  was  received  outside  the  military 
treatment  facility.  Although  this  project  showed  satisfaction  percentages  below  95%  (Q10  and 
Q1 1),  the  project  demonstrated  that  MOU  HNII  satisfaction  levels  of  U.S.  beneficiaries  when 
compared  to  the  121st  Combat  Support  Hospital  were  slightly  higher,  with  Q1 1  statistically 
significant  (x  =  13.3;  p  =  .001 ;  df=  2).  The  United  States  Department  of  Defense’s  Military 
Health  System  continues  to  be  placed  under  increased  public  scrutiny  and  pressure  to  provide  the 
Nation’s  sons  and  daughters  (along  with  their  families)  accessible  and  quality  healthcare. 
Although  cost  is  a  consideration  within  the  Military  Health  System,  the  delivery  of  always 
accessible  and  quality  healthcare  trumps  cost.  For  this  reason,  along  with  many  regulatory  and 
accreditation  guidelines,  monitoring  and  ensuring  beneficiaries  are  satisfied  with  their  healthcare 
has  become  an  indicator  which  is  closely  tracked  and  reported  at  all  levels  within  the  United 
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States  Government.  Military  members  and  their  families  will  continue  to  be  stationed 
throughout  the  world,  and  in  many  cases,  in  a  location  with  limited  healthcare  options  requiring 
host  nation  medical  support.  As  TRICARE  expands  with  future  managed  care  contracts,  close 
attention  will  be  made  to  develop  matrixes  within  those  contracts  to  monitor  beneficiary 
satisfaction  (regardless  of  where  care  is  delivered  or  received).  The  purpose  of  this  project  was 
not  to  determine  why  or  how  many  beneficiaries  received  care  outside  the  military  treatment 
facility,  although  this  is  an  area  of  concern.  Some  level  of  care  will  always  be  received  by  US 
beneficiaries  outside  the  military  treatment  facility  especially  in  the  overseas  environments  due 
to  resource  constraints  and  military  operational  tempo.  The  standard  of  care  (access)  promised 
to  each  beneficiary  has  remained  unchanged  and  appointments  within  the  military  treatment 
facility  continue  to  be  difficult  to  obtain,  as  such,  referrals  to  MOU  IINH  will  continue  to 
increase. 

Limitations  of  this  project  include  a  low  response  rate  of  21%  and  sample  size  (goal  25%  or 
greater).  Although  the  potential  impact  to  this  project  is  considered  extremely  low,  surveys 
returned  did  not  indicate  whether  or  not  beneficiary  was  an  inpatient  or  outpatient  (only 
outpatients  requiring  transportation  were  physically  provided  the  survey  via  the  TRICARE 
Service  Center  in  the  Seoul  area,  surveys  were  available  for  pick  up  at  both  the  TRICARE 
Service  Center  and  the  MOU  IINIIs  by  anyone).  There  exists  no  established  method  to  confirm 
that  every  referred  outpatient  received  a  surve\  outside  the  Seoul  TRICARE  Service  Center  area 
(some  patients  drove  themselves  and  may  not  have  picked  up  a  survey  at  the  MOU  HNH  or 
TRICARE  Service  Center);  the  TRICARE  Service  Center  database's  reliability  is  questionable 
in  determining  the  number  of  referred  outpatients  (the  system  was  designed  to  track 
supplemental  care  only  and  fields  are  not  clearly  defined);  Retirees.  Retiree  Family  Members  and 
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Others  (Department  of  State  and  Embassy  personnel,  Korean  service  employees  and  Department 
of  Defense  School  System  teachers)  are  able  to  seek  care  directly  at  MOU  HNH  without  going 
through  the  TRICARE  Service  Center  and  may  not  pick  up  (or  turn  in)  a  survey;  and  the 
beneficiary  category  “Other"  with  an  84%  overall  satisfaction  includes  all  ages  with  various 
backgrounds  making  it  extremely  difficult  to  narrow  down  potential  trends.  It  was  assumed  that 
each  beneficiary  that  received  a  HNH  Survey  was  properly  briefed  (without  bias)  and  understood 
the  process  to  complete  and  turn-in  the  survey.  It  is  assumed  that  SPSS  and  XLSTAT  produced 
objective  and  replicable  results. 

Although  it  was  believed  the  non-command  sponsored  beneficiaries  (which  are  required  to 
pay  for  healthcare  services  upfront  and  out  of  pocket)  would  have  a  much  lower  satisfaction 
level  when  compared  to  command  sponsored  beneficiaries,  the  results  did  not  support  the 
assumption  Command  sponsorship  and  the  beneficiary  categories  of  Active  Duty  and  Active 
Duty  Family  Member  represented  a  mere  2%  mean  difference  in  overall  satisfaction.  Korea 
(Asia  in  general)  has  a  male-dominated  culture,  with  more  respect  and  attention  given  to  the 
male  gender.  Results  supported  that  gender  for  the  U.S.  beneficiary  population  does  influence 
overall  satisfaction;  male  satisfaction  10%  higher  mean  than  females  (all  five 
questions/composite  average).  As  in  the  HNHs,  the  Korean  provider-U.S.  beneficiary 
relationship  was  expected  to  have  a  negative  influence  on  patient  satisfaction  amongst  female 
patients.  This  may  be  due  to  the  HNH  language  challenges  and  levels  of  acceptance  (and 
comfort  zones)  between  genders.  Healthcare  leaders  should  focus  attention  on  why  males  have  a 
higher  level  of  overall  patient  satisfaction  than  females.  The  IHO  should  build  a  focus  group  or 
root  cause  analysis  team  composed  of  beneficiaries  and  a  multi-disciplinary  hospital  team  with 
HNH  staff  to  examine  further.  Although  a  gender  distribution  of  providers  and  staff  was  not 
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conducted  within  this  project,  it  may  be  determined  that  gender  or  cultural  sensitivity  training 
may  prove  useful  for  the  beneficiaries,  the  IHO  staff  or  the  11NH  staff.  Another,  less  resource 
intensive  option,  may  be  to  develop  a  statistical  process  control  chart  with  the  MOU  HNH  survey 
data  to  determine  if  there  are  any  special  causes  for  variation.  As  seen  in  the  statistically 
significant  correlations  in  Table  5,  this  project  supports  findings  that  when  a  provider  listens,  the 
staff  is  helpful  and  continuity  of  care/coordination  is  smooth  satisfaction  scores  are  impacted. 
Additionally,  this  project  has  reinforced  the  need  to  maintain  constant  communication  (transfer 
of  information)  between  facilities,  providers  and  patients,  particularly.  In  the  overseas  (multi¬ 
culture)  environment,  especially  for  the  US  beneficiary,  it  will  be  mutually  beneficial  to 
strengthen  the  relationships  for  sharing  resources  and  information  between  the  IHO  and  host 
nation  facilities  related  to  patient  satisfaction. 

Patient  satisfaction  information  is  and  will  continue  to  be  an  important  source  of  information 
for  health  professionals,  insurers,  and  consumers  (Aday,  1996).  Healthcare  delivery  is  becoming 
more  and  more  dependent  on  patient  satisfaction  data  (this  includes  their  perceptions  and 
expectations)  and  has  become  a  strong  variable  in  future  organizational  strategic  planning. 

Using  this  information  may  assist  leaders  and  planners  in  deciding  how  and  where  to  better 
allocate  resources.  This  information  may  help  civilian  providers  to  determine  whether  or  not 
they  would  like  to  become  part  of  the  TRICARE  network  (providing  the  opportunity  was 
presented).  The  value  of  patient  satisfaction  studies  may  not  only  prove  beneficial  to  military 
healthcare  administrators,  military  providers  and  network  (civilian)  providers,  but  will  perhaps 
aid  in  the  consideration  of  future  managed  care  contracts  and  provider  network  development. 

For  all  healthcare  institutions,  patients  are  an  invaluable  source  of  information  that  cannot  be 
ignored,  no  matter  where  they  receive  care  or  service.  A  means  to  collect,  analyze  and  report 
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this  information  to  healthcare  leadership  tor  appropriate  action  and  planning  is  even  more  critical 
when  health  care  delivery  is  received  in  a  foreign  environment.  A  reliable  and  valid  tool  to  view 
U.S.  (outpatient)  beneficiary  satisfaction  within  the  ROK  medical  facilities  provides  an 
additional  planning  (decision  making  and  problem  solving)  resource  for  Department  of  Defense 
Military  Health  System  and  host  nation  leadership.  As  mentioned  previously,  the  accurate  and 
timely  measurement  and  analysis  of  patient  satisfaction  can  be  an  indicator  of  quality,  ultimately 
enhancing  the  care  of  patients,  which  is  the  core  purpose  of  healthcare. 

CONCLUSIONS 

Although  this  project  showed  US  outpatient  beneficiaries  seen  at  an  MOU  IfNH  have  a 
higher  mean  satisfaction  with  their  provider  and  facility  visited  when  compared  to  the  121st 
Combat  Support  Hospital  and  Camp  Zama  Medical  Activity,  there  remains  room  for 
improvement  (especially  in  meeting  the  DoD/MHS  target  of  95%  overall  satisfaction  with 
provider).  As  the  command  sponsorship  continues  to  increase  (current  FY08  USFK  policy  adds 
3,000  slots)  in  the  ROK,  the  number  of  family  beneficiaries  will  continue  to  grow.  This 
population  represents  a  greater  acuity  and  higher  healthcare  utilization  requirement  than  the 
active  duty  population.  The  IHO’s  primary  mission  is  healthcare  to  its  active  duty  population; 
this  may  increase  the  probability  of  higher  utilization  of  referrals  to  MOU/HNHs.  As  more  and 
more  resources  are  channeled  to  support  Operations  Enduring  Freedom  and  Iraqi  Freedom, 
TRICARE  access  to  care  standards  are  increasingly  scrutinized  by  the  public;  (can  the  military 
health  system  continue  to  take  care  of  the  Nation’s  sons  and  daughters  properly).  Patient 
satisfaction  targets  set  by  DoD/MHS  will  continue  to  evolve  with  the  eventual  tying-in  of 
financial  incentives  for  the  organizations  that  meet  established  goals. 
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RECOMMENDATIONS 

As  a  result  of  this  project  the  following  recommendations  were  made  and  implemented  to 
increase  response  rates:  modify  survey  distribution  and  collection  by  providing  the  IINII 
facilities  with  copies  of  the  survey  to  hand  out.  provide  drivers  at  all  locations  copies,  build  into 
the  TRICARE  Service  Center’s  standing  operating  procedures  a  short  brief  to  patients  (along 
with  a  blank  survey)  prior  to  transport  to  IINII.  offer  a  web-based  survey  option  and  have  the 
local  IHO  TRICARE  Marketing  Officer  advertise  and  offer  incentives  to  complete  the  surveys 
(e.g.,  token  gifts)  on  an  on-going  basis.  Additionally,  a  recommendation  was  made  to  coordinate 
with  Camp  Zama,  Japan  TRICARE  Service  Center  to  review  and  potentially  adopt  best 
practices. 

Future  studies  and  research  should  include  methodologies  to:  dissect  the  beneficiary  category 
“Other”  and  educate  both  the  US  beneficiary  population  and  MOU  HNH  provider  on  cultural 
differences  (with  special  focus  on  gender  expectations).  The  project  did  not  assess  the  gender  of 
the  providers,  recommend  including  this  in  future  research.  Recommend  tracking  the  data  for  a 
longer  period  to  increase  the  sample  size  and  potentially,  the  response  rate  (to  include  utilizing 
the  data  from  the  ICE  on-line  responses).  Recommend  the  Military  Health  System  broaden  the 
current  outpatient  satisfaction  survey  to  include  non-institutional  medical  treatment  facilities.  It 
is  critical  to  continue  outsourcing  (similar  to  APLSS  and  the  Service  Delivery  Assessment)  the 
requirement  for  data  collection  and  analysis;  this  provides  the  survey  and  statistical  expertise,  in 
turn  a  potentially  more  reliable  and  valid  survey.  Currently,  data  for  the  APLSS  is  pulled  on  a 
regular  basis  for  each  military  medical  treatment  facility  where  it  is  later  consolidated  and 
forwarded  to  a  contractor  for  statistical  analysis  and  reporting.  Currently,  the  Military  Health 
System  does  not  house  non-institutional  encounters  and  visits  in  the  APLSS;  however  a  database 
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could  be  developed  to  pull  data  from  military  health  systems  world-wide,  after  the  specific 
variables  and  fields  are  identified.  Developing  a  more  specific  means  to  measure  satisfaction 
with  actual  healthcare  outcomes  may  be  one  option  for  improvement.  This  could  be  as  simple  as 
adopting  the  US  Air  Forces'  Service  Delivery  Assessment  (SDA)  tool.  This  SDA  incorporates 
call  centers  throughout  the  US  who  call  patients  within  24  hours  of  their  visit  with  specific 
questions  focused  on  outcomes,  which  can  be  modified  at  various  levels  for  benchmarking, 
measurements  and  trending.  Results  of  the  SDA  are  posted  immediately  online.  Finally, 
recommend  the  IHO  increase  marketing  efforts  of  the  current  surveys,  along  wath  potential 
incentives  (e.g.,  T  RICARE  key-chain)  for  timely  participation. 
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Appendix  A:  Map  of  MOU  Host  Nation  Hospitals  in  the  ROK 
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Source:  18th  MEDCOM  command  brief  (Jolissaint,  2007). 
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Appendix  B:  Healthcare  Comparisons  between  South  Korea  and  United  States 


South  Korea 

United  States 

Indicator 

Value  (year) 

Value  (year) 

Source 

Physicians  (number) 

75,045  (2003) 

730,801  (2000) 

1 

Physicians  (density  per  1  000  population) 

1.57  (2003) 

2.56  (2000) 

1 

Nurses  (number) 

83,333  (2003) 

2,669,603 

(2000) 

1 

Nurses  (density  per  1  000  population) 

1.75  (2003) 

9.37  (2000) 

1 

Midwives  (number) 

8.728  (2000) 

463,663  (2000) 

1 

Midwives  (density  per  1  000  population) 

0.19(2000) 

1.63  (2000) 

1 

Dentists  (number) 

16,033  (2003) 

249,642  (2000) 

1 

Dentists  (density  per  1  000  population) 

0.34  (2003) 

0.88  (2000) 

1 

Total  expenditure  on  health  as  percentage  of 
gross  domestic  product 

5.5  (2004) 

15.4  (2004) 

1 

General  government  expenditure  on  health 
as  percentage  of  total  expenditure  on  health 

52.6  (2004) 

44.7  (2004) 

1 

Private  expenditure  on  health  as  percentage 
of  total  expenditure  on  health 

47.4  (2004) 

55.3  (2004) 

1 

General  government  expenditure  on  health 
as  percentage  of  total  government 
expenditure 

10.3  (2004) 

18.9(2004) 

1 

Social  security  expenditure  on  health  as 
percentage  of  general  government 
expenditure  on  health 

79.2  (2004) 

28.0  (2004) 

1 

Out-of-pocket  expenditure  as  percentage  of 
private  expenditure  on  health 

80.40  (2004) 

23.80  (2004) 

1 

Private  prepaid  plans  as  percentage  of 
private  expenditure  on  health 

7.1  (2004) 

66.4  (2004) 

1 

Per  capita  total  expenditure  on  health  at 
average  exchange  rate  (US$) 

776.9  (2004) 

6096.2  (2004) 

1 

Per  capita  government  expenditure  on  health 
at  average  exchange  rate  (US$) 

408.5  (2004) 

2724.7  (2004) 

1 

Hospital  beds  (per  10  000  population) 

66.0  (2002) 

33.0  (2003) 

1  ' 

Population 

49044790 

(2007) 

301,139,947 

(2007) 

2 

Gross  national  income  per  capita  (PPP 
international  $) 

21850  (2005) 

41950  (2005) 

1 

Density  (pop/sq.km) 

487 

31 

o 

J) 

Source:  1-  World  Health  Organization,  2006;  2-  South  Korea,  2007;  3-  2007  World  population 
data  sheet,  2007. 
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121st  COMBAT  SUPPORT  HOSPITAL 
ATTN:  EAMC-MC-TRICARE 
UNIT:  15244 
APO,  AP  96205-5244 
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Appendix  D:  Office  of  Surgeon  General  “Army  Patient  Satisfaction  Survey’' 


DEPARTMENT  OF  THE  ARMY 
OFFICE  OF  THE  SURGEON  GENERAL 
SURVEY  PROGRAM  OFFICE  (SUITE  609) 
5109  LEESBURG  PIKE 
FALLS  CHURCH.  VA  22041-3258 


oen  O'  Ir*  ot-K 

T*<r  juwe  3a« 

EXAMPLES. 

Cartel  ^correct 

X 

•*».  j*n  yoa*  co'no'rted  qjMt  omivt 
envuooe  to  »  3  Set*  iC-32  Cn  .  5 

Army  Patient  Satisfaction  Survey 

We  need  your  help.  We  arc  trying  to  improve  the  quality  of  care  we  give  our  Soldiers  and  their  families. 

According  to  our  records  you  recently  had  a  healthcare  visit  with  (PROVIDER'S  NAME)  on  (VISI  f  DAHL)  at  the 
I  ~~1  hospital  .  Is  this  correct? 


Yes . .  □  Please  continue  with  the  survey 

No,  saw  someone  else.  .  □  Please  continue  with  Q9 

No  didn't  have  visit . .  □  ->  Please  stop  and  return  your  survey  now 

Thinking  specifically  about  your  visit  with  iPROVIDLR  S  NAME?  on  [VISI  t  DA  I  L)  at  the  I  ~  I 

hospital  please  rate  how  much  you  disagree  or  agree  with  each  of  the  following  Please  mark  an  ”X’*  in  the  box 
for  the  answer  that  is  closest  to  your  opinion 


1. 

2. 

3. 

4. 

5. 

6 

7. 

3. 


Completely 

OiMiifee 

Sornewfut 

D’Mtxee 

*ie  Utter  Aijree 
nor  Disagree 

So  r  new  r>4i! 
Ap'ee 

Completely 

Apree 

This  provder.  (PROVIDER  S  NAME;,  spent  the  time  with  you 
that  your  medical  problem  required . . . 

1 

□ 

□ 

□ 

□ 

□ 

T  his  provider  listened  to  you  careluiiy  about 

your  concerns  and  questions . 

□ 

□ 

□ 

□ 

□ 

This  provider  understood  your  problem  or 

condition . 

.  □ 

□ 

□ 

□ 

□ 

This  provider  treated  you  with  courtesy  and 

respect  . 

.  □ 

□ 

□ 

□ 

□ 

Tms  provder  explained  what  was  being 

done  and  why . 

□ 

□ 

□ 

□ 

□ 

T  ms  provder  hoi  pod  you  with  your  problem . 

□ 

□ 

□ 

□ 

□ 

Completely 
Dissd.  sited 

Somew  flat 
Dik»4nfie4 

Me  A  her 

Sdtttfit*']  <K>r 

Dta&Af}  sfled 

Somewhat 

Srfsfierf 

Completely 

S*tt»fi«d 

Overall,  how  satisfied  do  you  feel  about  your  visit  with 

[PROVIDERS  NAME  ? . 

□ 

□ 

□ 

□ 

D 

Much  cl  the  following  best  desenbes  your  familiarity  with  [PROVIDERS  N  AIL  ■? 

This  pfovder  is  my  Primary  Care  Manager  (PCM)  whom  I  see  tor  most  ol  my  routine  care .  □ 

f  ms  provder  is  not  my  PCM.  Out  I  had  met  or  heard  of  ftrrVfier  before  this  visit .  □ 

This  provider  is  not  my  PCM.  I  had  a  relerraito  see  this  provider .  □ 

This  provder  is  not  my  PCM.  and  l  had  never  met  or  heard  of  hirrvncr  before  tms  visit .  ....  □ 


a> 


Please  turn  over  and  continue  on  tho  back  page 
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Appendix  D:  Office  of  Surgeon  General  “Army  Patient  Satisfaction  Survey”  (back) 


Please  tell  us  how  you  were  treated  by  staff  before  and  after  you  saw  the  healthcare  provider.  Still  thinking 
about  your  visit  with  (PROVIDLR  S  NAME)  on  (VISIT  DATE),  please  rate  the  following  aspects  of  your  care  and 
service  during  that  visit. 

P  iy.>r 

Fair 

G-xkI 

Very 

(1 

y.  T  he  overall  phone  service  you  received  in 
scheduling  the  appointment  for  this  visit . 

□ 

□ 

□ 

□ 

□ 

□ 

13.  How  well  your  needs  and  schedule  were 
taken  into  consideration  when  this 
appointment  was  scheduled . . . 

□ 

□ 

□ 

□ 

□ 

□ 

11.  The  amount  of  time  from  when  you  made 
the  appointment  until  you  actually  saw  the 
healthcare  provider . 

□ 

□ 

□ 

□ 

□ 

□ 

12.  The  amount  of  time  you  waited  at  me 

clinic  to  seethe  healthcare  provider . 

□ 

□ 

□ 

□ 

□ 

□ 

1  3.  Courtesy  and  helpfulness  of  the  stall 

during  this  visit  . 

□ 

□ 

□ 

□ 

c 

□ 

14.  The  coordination  among  nil  the  people 
who  cared  lor  you  during  this  visit . 

□ 

□ 

□ 

□ 

□ 

□ 

15.  The  cleanliness  ol  the  1  acuity  you 

visited . 

□ 

□ 

□ 

□ 

□ 

□ 

16.  The  corrdort  of  the  facility  you 

visited  . 

□ 

□ 

□ 

□ 

□ 

□ 

17.  The  convenience  of  the  facility  you 

visited  . 

□ 

□ 

□ 

□ 

D 

□ 

If  you  also  went  to  the  Pharmacy,  Laboratory  or  Radiology  Department  in  conjunction  with  your  visit 
please  rate  your  experience  with  these  services: 

on  (VISIT  DATE). 

*> 

!#<«*»• 

r  air 

V«*ry 

GooJ 

Eicelfeni 

18.  Overall,  how  would  you  rate  your  visit  to 
the  Pharmacy? . . . 

□ 

□ 

□ 

□ 

□ 

□ 

1 3.  Overall,  how  would  you  rate  your  visit  to 
the  Laboratory? . 

□ 

□ 

□ 

□ 

□ 

□ 

2D.  Overall,  how  would  you  rate  your  visit  to 
the  Radiology  Department?... . 

□ 

□ 

□ 

□ 

□ 

□ 

Do  you  have  any  comments  about  your  visit  with  PROVIDER  S 

NAME'*  on  i 

VI 51 1  DATE: 

21.  Everything  considered,  hew  satisfied  were  you  withL 

1  hospital 

during  this  visit? 

CorriDlet?lv  Dissatisfied  Somewhat  Dissatisfied 

Neither  Satisfied  nor  Dissatisfied  Somewh at  Satisfied  Completely  Satisfied 

□ 

□ 

□ 

□ 

□ 

Thank  you  very  much  for  your  opinions. 

Please  return  this  survey  today  in  the  self-addressed  envelope. 

ATTN  AMEDO  SURVEY  CENTER 

P.0  BOX  5033 

CHICAGO.IL  60680 
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Appendix  E:  Interactive  Customer  Evaluation  Host  Nation  Referral  Comment  Card 


E 


ie 

_ 

Interactive 

Home  •  About  ICE  ■  Manager  Login  -  Help 

£ 

Customer 

Evaluation 

Service  Provider  Search:  5? 

Home  »  Yonqiw  Army  Post  »  Health  »  Comment  Card 


121st  Genera  I  Hospital/Host  Nation  Referral  Comment  Card 

Welcome  to  the  18th  MEDCOM  Interactive  Customer  Evaluation  system.  We  welcome  any  Compliment  or  Concern  that  you  may  have 
for  the  121st  General  Hospital. 

People  don’t  C3re  how  much  you  know,  until  they  know  how  much  you  C3re. 

InforrrMion  about  thl*  »ervke  providor  (I  AQ»  LvenU  Contact*. Link*) 

Customer  Service: 


Facility  Appearance 

O  Excellent 

O  Good 

O  OK 

O  Poor 

O  Awful 

©  N/A 

Employee/Staff  Attitude 

O  Excellent 

O  Good 

0  OK 

O  Poor 

O  Awful 

©  N/A 

Timeliness  of  Service 

O  Excellent 

O  Good 

O  OK 

O  Poor 

O  Awful 

®  N/A 

Hours  of  Service 

O  Excellent 

O  Good 

O  OK 

O  Poor 

O  Awful 

©  N/A 

Did  the  product  or  service  meet  your  needs? 

O  Yes  O  No 

©  N/A 

Your  provider  listened  to  you  carefully 
about  your  concerns  and  questions. 

O  Excellent 

o 

Good 

O  OK 

O  Poor 

O  Awful  ©  N/A 

The  provider  understood  your  problem 
or  condition 

O  Excellent 

o 

Good 

O  OK 

O  Poor 

O  Awful  ©  N/A 

The  provider  treated  you  with 
courtesy  and  respect 

O  Excellent 

o 

Good 

O  OK 

O  Poor 

O  Awful  ©  N/A 

The  provider  explained  what  was 
being  done  and  why 

O  Excellent 

o 

Good 

O  OK 

O  Poor 

O  Awful  ©  N/A 

The  amount  of  time  you  waited  at  the 
clinic  to  see  the  healthcare  provider 

O  No  Experience 

o 

Poor 

O  Fair 

O  Good 

O  Very  Good  O 
Excellent  ©  N/A 

Courtesy  and  helpfulness  of  the  staff 
during  this  visit 

O  No  Experience 

o 

Poor 

O  Fair 

O  Good 

O  Very  Good  O 
Excellent  ©  N/A 

The  coordination  among  all  the  people 
who  cared  for  you  during  this  visit 

O  No  Experience 

o 

Poor 

O  Fair 

O  Good 

O  Very  Good  O 
Excellent  ©  N/A 

The  cleanliness  of  the  facility  you 
visited 

O  No  Experience 

o 

Poor 

O  Fair 

O  Good 

O  Very  Good  O 
Excellent  ©  N/A 

The  comfort  of  the  facility  you  visited 

O  No  Experience 

o 

Poor 

O  Fair 

O  Good 

O  Very  Good  O 
Excellent  ©  N/A 

Everything  considered,  how  satisfied 
were  you  with  the  facility  dunng  your 
visit 

Overall,  how  satisfied  do  you  feel 
about  your  visit  with  your  provider 

Please  indicate  your  status 

Please  indicate  your  gender 

Are  you  Command  Sponsored 

Name  of  Facility  or  Hospital  you  visited 


O  Completely  Dissatisfied  O  Somewhat  Dissatisfied  O  Niether 
Satisfied/Dissatisfied  O  Somewhat  Satisfied  O  Completely  Satisfied 

©  N/A 

O  Completely  Dissatisfied  O  Somewhat  Dissatisfied  O  Niether 
Satisfied/Dissatisfied  O  Somewhat  Satisfied  O  Completely  Satisfied 

®  N/A 

O  Active  Duty  O  Family  Member  -  Active  Duty  O  Retired  O 

Family  Member  -  Retired  ©  N/A 

O  Male  O  Female  ©  N/A 

O  Yes  O  No  ©  N/A 
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Appendix  F:  Information  Paper  on  Appointment  Shortages 

U.S.  Army  Hospital-Yongsan 
Appointment  Shortage  Information  Paper 

The  United  States  Army  Hospital-Yongsan  (USAH-Y)  experienced  a  notable  decrease  in  available  patient 
appointments  during  Oetober  -  December  2007  and  January  2008;  this  shortage  specifically  affected  adult 
Primary  Care,  Neurology,  Dermatology,  Allergy,  Podiatry,  Obstetries/Gyneeology  and  the 
Behavioral  Health  Clinics.  In  the  past  year,  most  clinics  were  able  to  take  care  of  all  beneficiary 
categories  and  did  not  need  to  prioritize  appointments  based  on  beneficiary  status  (NOTE;  behavioral 
health  has  been  limited  to  TRICARE  Prime  for  the  past  two  years).  Due  to  the  current  shortage  of 
available  appointments  and  our  inability  to  meet  TRICARE  access  standards,  we  will  now  begin  to 
prioritize  access  for  some  routine  and  specialty  care  appointments. 

The  United  States  Army  Hospital-Yongsan  access  priorities  will  follow  DoD  guidelines  and  are  as 
follows: 

1.  TRICARE  Prime  Active  Duty  Service  Members 

2.  TRICARE  Prime  Active  Duty  Family  Members 

3.  TRICARE  Plus  (enrolled  non-eommand  sponsored  Active  Duty  Family  members  and 
grandfathered  Retirees  and  their  Family  Members) 

4.  Other  Active  Duty  Family  Members  (TRICARE  Standard) 

5.  Other  Retirees  and  their  Family  Members  (Non-TRICARE  Plus) 

6.  All  other  beneficiaries  (Pay  Patients  —  GS,  NAF,  DoDDS  employees,  etc.) 

Priority  5  and  6  patients  will  be  offered  primary  care  appointments  on  a  spaee-available  basis  until  the 
affected  elinies  reach  normal  capacity  and  have  the  available  appointments  to  accommodate  the  entire 
patient  population.  Priority  5  retirees  and  their  family  members  may  be  booked  into  unfilled  routine  or 
specialty  appointments  up  to  one  week  prior  to  the  appointment  Priority  6  patients  may  be  booked  into 
unfilled  routine  or  specialty  appointments  up  to  72  hours  prior  to  the  appointment. 

Telephone  renewals  of  current  medications  may  be  possible  for  certain  patients;  but  if  a  elinic  visit  is 
required  to  fill  or  refill  medications,  then  Priority  5  and  6  patients  may  be  required  to  make  an  appointment 
with  a  physician  at  an  outside  facility  (preferably  a  Korean  MOU  Hospital)  unless  a  spaee-available 
appointment  becomes  available. 

Additional  clinical  providers  have  been  hired,  and  they  are  now  on  board  and  working  in  our  Primary  Care 
elinies;  more  providers  will  be  hired  m  the  near  future.  Wc  anticipate  this  appointment  shortage  will  ease 
by  April  2008  as  additional  providers  are  integrated  into  the  hospital  staf£  and  our  existing  backlog  of 
appointments  is  eliminated.  Once  we  have  a  sufficient  number  of  appointments  to  meet  demand,  our 
hospital  PAO  will  inform  the  community. 

The  18,h  Medical  Command  has  agreements  with  many  host  nation  medieal  facilities.  Patients  who  are 
unable  to  obtain  an  appointment  may  obtain  primary  care  and  specialty  care  appointments  from  one  of 
these  Korean  MOU  Hospitals.  Those  beneficiaries  who  choose  to  utilize  this  option  may  visit  the  18th 
MEDCOM  MOU  Hospital  webpage  at  http://www.seoul.amedcj.annv.mil/sites/mou  hosp/index.htm  for  the 
list  of  eurrent  USFK/1 8th  MEDCOM  Korean  MOU  Hospitals.  Non-TRICARE  patients  should  expect  to 
pay  up-front  fees  for  the  care  they  reeeive  in  MOU  Hospitals,  and  TRICARE  standard  patients  should 
expect  to  pay  their  eo-pay  up-front 

We  truly  apologize  that  we  are  required  to  resort  to  these  drastic  measures  in  order  to  provide  care  to  our 
DoD  beneficiaries,  and  your  patience  is  appreciated  as  we  grow  our  workforce  to  meet  the  demands  of 
USFK  throughout  the  Korean  Peninsula.  It  is  our  primary  mission  and  goal  to  provide  quality,  safe,  and 
accessible  earc  to  our  patients  —  and  we  are  optimistic  that  we  will  be  able  to  restore  our  open  aceess  in  the 
near  future. 


Approved  by  COL  Greg  Jolissaint 
Commander,  USAH-YongSan 
As  of  25  January  2008 
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Appendix  G:  APLSS  Validity  Information  Paper  from  Synovate  Company  (1/6) 


Validity  of  the  APLSS  Survey  (Svnovate  Healthcare,  2007) 

Validity  in  survey  instruments  is  the  property  that  they  measure  consistently  the  domains 
of  content  that  they  are  intended  to  measure.  Validity  is  not  typically  represented  by  a 
single  coefficient,  but  is  indicated  instead  by  an  accumulation  of  evidence  that  the  items 
and  the  summary  indices  derived  from  them  make  sense  (face  validity),  produce  scores 
that  discriminate  among  persons  or  objects  known  to  differ  (discriminant  validity),  are 
stable  over  time  (reliability  or  internal  consistency),  respond  appropriately  to  changes  in 
the  underlying  environment  (dynamic  validity),  and  are  correlated  with  other  measures 
that  they  should  based  on  theory  and  experience  (concurrent  validity).  To  that  could  be 
added  actionability,  or  practical  validity,  which  is  the  usefulness  of  the  information  in 
guiding  effective  action.  The  APLSS  and  WTU  surveys  are  discussed  below  with  respect 
to  these  indicators  of  validity. 

APLSS  Survey 

The  APLSS  questionnaire  was  designed  to  meet  the  following  objectives: 

>  Measure  overall  satisfaction  with  the  provider  and  overall  satisfaction  with  visit 
including  provider  and  all  services. 

>  Provide  drivers  of  quality  measures  for  improving  the  interaction  between 
providers  and  their  patients. 

>  Monitor  performance  for  key  aspects  of  aceess  and  service  provided  by  facilities. 
The  validity  of  the  APLSS  questionnaire  can  be  evaluated  by  the  following  indicators: 

>  The  questionnaire  measures  actionable  dimensions  of  quality  and  service  for 
AMEDD  outpatient  services,  and  in  particular  for  the  interaction  between  the 
provider  and  patient. 

>  Questionnaire  dimensions  and  survey  wording  are  based  on  standards  found  in  the 
healthcare  services  research  industry. 

>  Questionnaire  document  circulated  to  managers  within  the  MHS  and  to  users  of 
the  MHS  for  evaluation  of  question  wording  and  actionability  of  information. 

>  Quantitative  evaluation  of  effectiveness  of  drivers  of  satisfaction  and  the  effect  of 
data  on  provider  and  MTF  performance. 

Synovate  used  existing  survey  programs  to  identify  key  aspects  of  service  and  drivers  of 
provider  performance.  The  existing  survey  programs  included  the  Consumer  Assessment 
of  Health  Plans  Survey  (CAHPS)  questionnaire,  the  Kaiser  Permanente  Northern 
California  Patient  Satisfaction  Survey  questionnaires  and  the  Kaiser  Hawaii  Patient 
Satisfaction  Survey  questiomiaires.  These  questionnaires  were  designed  and  evaluated 
using  classical  survey  design  techniques  including  cognitive  evaluation,  plus  they  have 
been  used  effectively  in  existing  health  organizations  for  years.  Kaiser  Permanente  has 
been  able  to  use  survey  results  within  their  organization  to  effectively  promote  change 
and  improvement,  and  the  resulting  improvements  have  been  reflected  in  changes  in  their 
industry-wide  measures  such  as  CAHPS. 
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Synovate  drafted  APLSS  survey  questions  along  with  the  key  dimensions.  The 
dimensions  were  presented  to  AMEDD  staff,  question  wording  were  standardized 
according  for  to  scale  purposes,  and  a  formatted  questionnaire  was  drafted.  The 
questionnaires  were  circulated  among  AMEDD  managers  and  users  of  AMEDD.  This 
included  staff  of  the  OTSG  Decision  Support  Center  and  at  development  sites  of  Fort 
Benning  and  Fort  Bliss.  Suggestions  were  received  and  discussesd  among  managers  and 
users.  The  discussions  included  these  topics:  Can  the  results  of  this  question  be  used  to 
effect  change  or  improvements  in  your  facility?  In  your  experience  as  a  patient,  what  do 
you  think  the  question  is  asking,  and  can  you  answer  it  using  its  current  wording?  This 
review  eliminated  ambiguous  wording  and  irrelevant  questions. 

The  survey  results  have  undergone  analytical  review  at  different  stages  of  the  survey 
program.  This  included  the  pilot  stage  of  the  study  in  which  a  relationship  analysis  was 
conducted  to  assess  whether  the  survey  instrument  can  distinguish  between  high  and  low 
performing  providers.  This  analysis  looked  at  the  relationships  between  overall  scores 
and  attribute  scores  for  each  provider.  The  unit  of  analysis  is  the  provider  itself.  The 
relationships  seen  in  this  analysis  are  provided  in  Figure  1 .  Figure  1  shows  that  providers 
performing  low  on  each  dimension  have  low  satisfaction.  The  relationships  were  not 
uniform  suggesting  some  were  better  drivers  than  others,  but  the  relationships  were 
consistent  for  both  Fort  Benning  and  Fort  Bliss.  For  all  drivers  analyzed,  the  correlations 
were  very  high  (>.8),  indicating  substantial  validity.  These  findings  replicate  similar 
relationships  between  these  drivers  and  overall  satisfaction  found  in  surveys  in  the 
civilian  heathcare  services  industry. 
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Figure  1 :  Relationship  Analysis  for  Key  Drivers  of  Provider  Performance 
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A  second  analysis  was  conducted  after  the  full  roll  out  of  the  APLSS  program.  This 
analysis  looks  at  whether  respondent  using  the  survey  instrument  can  differentiate 
between  drivers  of  provider  satisfaction  versus  drivers  of  overall  satisfaction.  In  this 
analysis,  Synovate  conducted  an  analysis  called  attributable  effects.  This  analysis  looks 
at  attributes  that  drive  potential  gain  in  satisfaction  and  drivers  of  potential  loss.  The 
driver  with  potential  gain  may  not  be  the  same  as  a  driver  of  potential  loss.  This 
separates  out  drivers  that  one  should  focus  to  improve  scores,  but  it  also  identifies 
attributes  that  the  performance  should  be  maintained.  The  results  are  presented  in  Figure 
2.  Attributes  associated  with  access  and  service  were  not  drivers  of  satisfaction  with 
provider  satisfaction;  however,  drivers  of  overall  satisfaction  included  attributes 
associated  with  access  and  service  along  with  provider  attributes.  Specifically,  it  shows 
that  improvement  with  the  Attribute  “Helps  with  Problem”  will  the  greatest  improvement 
in  provider  satisfaction.  This  agrees  with  the  previous  analysis.  Tlie  analysis  picks  up 
differently  on  one  dimension  -  “Courtesy  and  Respect.'4  The  relationship  analysis  shows 
that  this  is  not  as  important  an  attribute  as  the  others,  but  the  attributable  effects  analysis 
shows  that  it  has  the  greatest  potential  loss.  That  is,  providers  that  treat  their  patients 
without  courtesy  and  respect  will  have  patients  with  low  satisfaction.  That  intuitive  result 
indicates  the  instrument’s  ability  to  capture  these  differences. 

The  attributable  effects  for  overall  satisfaction  in  Figure  3  shows  that  attributes  and 
provider  satisfaction  do  contribute  to  the  overall  satisfaction,  but  the  aspects  of  service 
and  access  have  even  greater  potential  gain.  This  indicates  that  the  instruments  and  the 
respondents  understand  the  differences  between  satisfaction  and  service  provided  by  the 
provider  and  satisfaction  and  service  provided  by  the  facility. 

Figure  2 
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Figure  3 
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A  final  analysis  about  the  underlying  effectiveness  of  the  survey  program  is  to  look  at  the 
change  provider  performance  after  they  start  receiving  feedback.  This  analysis  examines 
the  performance  of  providers  from  start-up  through  several  months  into  the  APLSS 
program-  An  effective  program  provides  feedback  and  improvement  should  be  seen.  For 
providers  performing  better  than  average,  they  should  maintain  their  scores.  If  so,  it 
show  the  instrument  and  the  program  is  doing  as  expected.  The  new  provider  analysis  in 
Figure  4  shows  performance  in  the  first  3  months  of  the  program.  The  providers  are 
grouped  by  the  lowest  performers,  next  lowest,  middle  performer  and  top  performers. 

The  results  show  that  the  worst  performers  have  substantial  improvement  over  the  course 
of  the  first  three  quarters,  and  the  next  lowest  and  middle  group  show  some  improvement, 
'lire  top  group  shows  a  slight  decline  in  performance,  but  they  still  score  very  high. 


Figure  4 
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The  development  process  of  the  questionnaires  along  with  the  subsquent  analyses  all 
validate  the  questionnaires  and  the  information  being  collected.  In  addition,  the  program 
provides  qualitative  feedback  in  addition  to  the  quantitative  measures.  The  mail  and  on¬ 
line  version  allows  respondents  to  give  written  comments  on  their  provider’s  service. 
These  comments  are  given  directly  to  the  provider  on  the  web-site.  The  survey  program 
provides  substantial  qualitative  feedback  about  access  and  service.  The  on-line  version 
prompts  respondents  for  reasons  they  responded  “fair”  or  “poor”  on  the  survey.  This 
provides  input  on  the  specific  reasons  for  a  low  performance  rating.  These  comments  are 
sent  directly  to  managers  and  commanders  weekly  via  e-mail  and  they  are  posted  on  the 
web-site.  This  along  with  the  overall  measure  for  that  attribute  establishes  a  magnitude 
of  the  problem,  and  it  provides  exact  causes  of  low  scores. 
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Table  7. 

Frequency  Distribution  for  Facility  Referred. 

Frequency 

Percent 

Cumulative  Percent 

Dankook 

49 

43 

43 

Dongsan 

30 

26.3 

69.3 

Samsung 

35 

30.7 

100 

Total 

114 

100 

Table  8. 


Frequency  Distribution  for  Beneficiary  Category. 


Frequency 

Percent 

Cumulative  Percent 

AD 

69 

60.5 

60.5 

ADFM 

36 

31.6 

92.1 

RET 

2 

1.8 

93.9 

RETFM 

2 

1.8 

95  6 

Other 

5 

4.4 

100 

Total 

114 

100 

Table  9. 


Frequency  Distributjqn  for  Gender. 


Frequency 

Percent 

Cumulative  Percent 

Female 

61 

53.5 

53.5 

Male 

53 

46.5 

100 

Total 

114 

100 

Table  10. 

Frequency  Distribution  for  Command^  Sponsorship 

Frequency  Percent  Cumulative  Percent 


No 

19 

16.7 

167 

Yes 

95 

83.3 

100 

Total 

114 

100 

Table  11. 


Frequency  Distribution  for  Month  Care  Received 


Frequency 

Percent 

Cumulative  Percent 

September 

30 

26.3 

26.3 

October 

41 

36 

62.3 

November 

43 

37  7 

100 

Total 

114 

100 
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Table  12. 

Frequency  Distribution  for  Q1 -Your  provider  listened  to  you  carefully  about  your  concerns  and 
questjons.  _  _ _ 

Frequency  Percent  Cumulative  Percent 


Score  1 ,  2  or  3 

12 

10.5 

10.5 

Score  of  4  or  5 

102 

89.5 

100 

Total 

114 

100 

Table  13 

Frequency  Distribution  for  Q6-Courtesy^  and  helpfulness  of  the  staff  during  thjs  visit 

Frequency  Percent  Cumulative  Percent  


Score  1 ,  2  or  3 

17 

14.9 

14.9 

Score  of  4  or  5 

97 

85.1 

100 

Total 

114 

100 

Table  14 

Frequency  Distribution  for  Q7-The  coordination  among  all  the  people  who  cared  for  you  during 
this  visit. 

Frequency 

Percent 

Cumulative  Percent 

Score  1 ,  2  or  3 

16 

14 

14 

Score  of  4  or  5 

98 

86 

100 

Total 

114 

100 

Table  15 

Frequency  Distribution  for  QlO-Overall,  how  satisfied  do  you  feel  about  your  visit  with  your 
provider.  _  _ _ _ _ 

Frequency  Percent  Cumulative  Percent  


Score  1 ,  2  or  3 

9 

7.9 

7.9 

Score  of  4  or  5 

105 

92  1 

100 

Total 

114 

100 

Table  16. 

Frequency  Distribution  for  Q1 1  -Everything  considered,  how  satisfied  were  you  with  the  facility 
during  your  v[sit. _ _ _  ___  _ 

Frequency  Percent  Cumulative  Percent 


Score  1 ,  2  or  3 

8 

7.1 

7  1 

Score  of  4  or  5 

108 

92.9 

100 

Total 

114 

100 
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- Original  Message - 

From:  Leighnor,  Agatha  Ms  TAMC 

Sent:  Tuesday,  August  14,  2007  12:25  PM 

To:  Hayman,  Alex  L  MAJ  121  Combat  Support  Hospital 

Subject:  RE:  Information  Request  (UNCLASSIFIED) 

Classification:  UNCLASSIFIED 

Caveats:  NONE 

MAJ  Hayman, 

Our  HUC/IRB  Acting  Chair,  COL  Donald  Person,  reviewed  your  description  with 
attached  new  survey  and  concurs  with  my  interpretation. 

Please  maintain  a  copy  of  this  correspondence  as  written  confirmation  that 
your  new  survey  is  not  research.  As  such,  the  survey  is  not  subject  to 
review-and-approval  by  the  TAMC  HUC/IRB. 

Aloha, 

Agatha 

- Original  Message - 

From:  Leighnor,  Agatha  Ms  TAMC 

Sent:  Tuesday,  August  14,  2007  5:42  AM 

To:  Hayman,  Alex  L  MAJ  121  Combat  Support  Hospital 

Subject:  RE:  Information  Request  (UNCLASSIFIED) 

Importance:  High 

Classification:  UNCLASSIFIED 

Caveats:  NONE 

Hello  MAJ  Hayman, 

Thank  you  for  your  inquiry.  I  am  including  LTC  Lund  on  this  email.  He  is 
the  Human  Protections  Administrator  for  the  18th  MEDCOM  and  will  be  pleased 
to  know  that  we  all  want  to  abide  with  appropriate  research  guidelines.  TAMC 
partners  with  the  18th  MEDCOM  as  their  IRB  of  record. 

The  activity  described  places  emphasis  on  internal  purposes/improvement 
rather  than  to  develop  or  contribute  to  generalizable  knowledge  (research) . 
Too,  it  does  not  involve  intervention  or  interaction.  I  see  no  reason  for  it 
going  through  the  IRB  but  will  solicit  opinions  from  others  then  get  back 
with  you. 

Aloha , 

Agatha 


